Projects
Traffic Congestion Control Using Reinforcement Learning:
Dr. Yufang Jin
Project description
This project aims to develop a simulation package for autonomous driving and route redirection in a designated region using reinforcement learning (RL) algorithms. The developed RL algorithms will determine the motion and routes of vehicles considering the traffic safety, shortest traveling path, shortest traveling time, and traffic conditions to reduce traffic congestion. We will further verify the algorithms using a hardwareintheloop testbed including scaledown tracks, carlike rovers, and traffic signaling systems. Currently, 3 rovers (75:1 scaledown) have been built, and traffic signaling systems are developed by a senior design group.
Tasks
 Learn the basic concept of the reinforcement learning algorithm; (Weeks 12)
 Get familiar with existing RL and interpretation algorithm developed in Python; (Weeks 23)
 Get familiar with Simulators including CARLA, SUMO, and Python programming (Weeks 23)
 Determine possible reward functions to optimize an objective function related to the motion of vehicles. (Weeks 46)
 Demonstrate singlevehicle motion or multiple vehicle routing with RL (Weeks 69)
 Write a weekly report, project progress, manuscript for paper submission, and REU final presentation; (Weeks 210)
Prerequisite
The basics of neural networks and Python are needed.
References:
 Introduction to Reforcement Learning by David SIlver: https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDMOYHWgPebj2MfCFzFObQ
Magma 5G Open Source Software Technology:
Dr. Brian Kelley
GRA:
Taylor Eisman
Project Description
The goal of the project is to learn next generation 5G SW Standards, 5G Programming, and 5G Networks critical to engineers and vital for emerging software engineers, network engineers, telecomm, and advanced R&D. Learn, download, and build models of 5G Networks used in the most advanced network standard in the world. REU Students compose 5G Network models from Open Source Libraries; develop a Protocol Data Unit Session for 5G Systems. Magma is a 5G Open Source Software project develop software applications that communicate 5G data packets. Facebook has developed Magma (see https://www.magmacore.org/). It is an open source software project for 5G Telecommunication researchers, operators, and labs for rapid prototyping of 5G Core Networks. The Open Source Project provides softwarecentric, distributed data packet networks, containerized network function, and tools for automated network management.
Tasks
 Install open source software packages from Anaconda, Pycharm, Magma, and Github
 Review Software tutorials with your mentors, once a week Download and Open Source 5G Software, Anaconda, Magma, and Github
 Set up software platforms and review software methodology with GRA mentor
 Students Develop 5G network prototypes in Open Source Software
Prerequisite
Basic programming in Matlab/Python
Construction of HomogeneityInspired Neural Network:
Dr. Chunjiang Qian
Project Description
Dynamic systems use mathematical models to describe various things or environments, such as unmanned vehicles, energysaving and SpaceX rockets. In order to simplify the modeling and control problems in dynamical systems, many big data technologies have been applied, and neural networks are one of them. Neural network has flexible architecture and can be very useful in classifications, regressions as long as enough data sets are collected. Neural networks are mainly used in the field of computer vision, but it can also be used to simulate dynamic systems and predict the future state of the system. Recent work on Neural Ordinary Differential Equations (Neural ODEs) [1] has provided a correspondence between discrete deep learning networks and continuoustime ODEs. This has given new tools for designing the network infrastructure of deep learning networks, as well as for training. [2] proposed an algebra framework and an approximation theory to reveal the relationship between the neural networks and the structure of the compositional function to be approximated. The compositional functions use directed acyclic graph (DAG) to represent a mapping between input and output and provide an intuitive structure. With this theoretical foundation, we can make error analysis of neural networks for functions as inputoutput relations and functions generated based on the trajectories of differential equations.
Tasks
 Use neural networks to simulate dynamic systems and predict the future state of them
 Construct a discretedepth network by employing the homogenous functions at the nodes. The homogeneous network (Figure 1) is represented as a directed acyclic graph (DAG) and can be considered as a compositional function [2]
 Realize the new construction and training in Python/MATLAB.
 Test different constructions of new neural networks and compare the effectiveness to the traditional construction.
Prerequisite
The basics of neural networks and Python/MATLAB are needed.
References:
 T.Q. Chen, Y. Rubanova, J. Bettencourt, D.K. Duvenaud, “Neural ordinary differential equations.” Advances in neural information processing systems, 2018.
 W. Kang, and Q. Gong, “Neural Network Approximations of Compositional Functions with Applications to Dynamical Systems.” arXiv preprint, arXiv:2012.01698, 2020.
Robotic Behavior Learning From Human Demonstrations: Dr. Yongcan Cao
Project Description
Learning complex behaviors is a challenging task for robots to demonstrate humanlevel performance in realworld environments. The objective of the project is to learn robotic behaviors through human guidance via learning from demonstrations. One important approach to enable robotic behavior learning is through inverse reinforcement learning, which is then followed by reinforcement learning. In particular, human demonstrations will be used to learn rewards, through inverse reinforcement learning, which can be used to generate control policies.
Tasks
 Setting up simulation environments
 Generating human demonstrations
 Adapting and executing inverse reinforcement learning algorithms
 Adapting and executing reinforcement learning algorithms
 Statistical Analysis
Prerequisite
 Strong background in programming, preferably in Python
 Strong background in statistics and control systems
 Excellent communication skills
 Prior experience on machine learning and/or robotics
Optimal Charging Strategy for AIpowered EVs:
Dr. Sara Ahmed
Project Description
The number of Electric Vehicles (EV) has increased significantly in the past decades due to its advantages including emission reduction and improved energy efficiency. However, the adoption of EV could lead to overloading the grid and degrading the power quality of the distribution system. It also demands an increase in the number of EV charging stations. To meet the charging needs of 15 million EVs by the year 2030 with limited charging stations, prediction of charging needs and reallocating charging resources are in emerging needs. Our group developed three modeling methods, multistep LSTM, onestep LSTM, and ARIMA to predict charging loads with temporal profiles for 3 charging stations in two years on the campus of the University of Texas at San Antonio.
In the summer REU program, an undergraduate student will help validate our predeveloped prediction models by developing a simulation platform and a proof of concept prototype that includes emulated smart charging systems and miniEV vehicle prototypes that would communicate together and host our proposed management system.
Tasks
 Learn the developed prediction models; (Weeks 12)
 Prepare the proofofconcept simulation/prototype; (Weeks 23)
 Use part of UTSA geoinformation (roadmap) as our simulation platform with entry, traffic flow information, intersection of roads, etc.
 Define number of EVs in the simulated study
 Define number and location of the EV charging stations, current utility ratio, price charging
 Develop an application that will input data to the cloud and provide the availability of the charging station; (Weeks 34)
 Demonstrate the proposed management system performance with computer simulations; (Weeks 46)
 Validate the system with the proofofconcept prototype; (Weeks 69)
 Write weekly report, project progress, manuscript for paper submission, and REU final presentation; (Weeks 210)
Prerequisite
The basics of neural networks and Python/MATLAB are needed. Experience with App development, hardware and software experience with Arduino and Raspberry Pi.
Understanding organically organized system functions and structures using artificial intelligence approaches:
Dr. Michelle Zhang
Project Description
Currently, most systems are assumed to have a causal system structure that isolate a set of factors that can be treated as “causes” and the rest as “effects” so that various phenomena can be interpreted [1], and inferences can be made for the effective intervention of such systems. However, such an approach often fails in practice because in reality, most dynamic systems are organized organically in which, parts are mutually casual [2]. For example, within the human body, if we take the view that the heart is the root organ and it affects the lung but not vice versa, then it’s treated as a causally organized system. However, if we take the view that all major organs are affecting each other mutually, then it’s an organically organized system. There lacks development of theories and approaches for inferring components and functions of such systems. The significance of the development is immense because virtually all medically related data inferencing problems should be modeled in the proposed approach.
Tasks
 Learn the basic concept of organically organized networks and develop simulated models of such networks using Matlab; (Weeks 12)
 Learn the basic concept of deep learning algorithm in Python and Matlab; (Weeks 23)
 Apply the existing algorithm to train the deep learning model with the simulated data; (Weeks 35)
 Apply the model for making inference of the learned model interpretation; (Weeks 69)
 Write weekly report, project progress, manuscript for paper submission, and REU final presentation; (Weeks 210)
Prerequisite
The basics of neural networks and Python/MATLAB are needed.
References:
 Pearl, Judea. “Causal inference.” Causality: Objectives and Assessment (2010): 3958.
 Hegel, G. W. F. (1929). Hegel’s science of logic.
Integrating onroad data for drivingsafety:
Dr. Miltos Alamaniotis
Project description
On the main features that most of the autonomous cars share is modern systems for the avoidance of collisions. Though systems have greatly improved in that direction, the majority of avoidance collision systems focus on cartocar crashes neglecting the cartoanimal collision. The objective of this project is to develop a system that predicts the behavior of animals on the fly. In that direction, deep learning with image processing will be utilized to integrate data taken from car sensors on the road. The Applied Artificial Intelligence Laboratory (directed by Dr. Alamaniotis) has novel datasets that will be used in the project. The student will have to utilize the datasets and develop the deep learning tool.
Tasks
 Data preparation (select datasets and create training and testing datasets)
 Develop of a deep network architecture
 Compare with a simple network (non deep)
 Validation of the tool
 Write Report
The student will have weekly meeting with the mentor (Dr. Alamaniotis) and will collaborate with grad students from the Applied Artificial Intelligence Laboratory.
Prerequisite
 Strong background in programming, preferably in Python/Matlab
 Knowledge of basic probabilities and statistical concepts
References:
 Munian, Y., MartinezMolina, A., & Alamaniotis, M., “A Design and Implementation of a Nocturnal Animal Detection Intelligent System in Transportation Applications,” ASCE International Conference on Transportation & Development, June 69, 2021, Virtual Conference, pp. 12.
 Munian, Y., Alamaniotis, M., MartinezMolina, A., “Intelligent System for Detection of Wild Animals Using HOG and CNN in Automobile Applications,” 9th International Conference on Information, Systems and Applications (IISA), Piraeus, Greece, July 2020, pp. 8.
Building Deep Learning Models that Understand Biology:
Dr. Yufei Huang
Project description
The evergrowing genomics data from omics profiling and singlecell technologies have spurred rapid developments of deep learning (DL) methods for genomics data analysis. While producing accurate prediction is a critical objective, the next generation omics analysis tools call for XDL models capable of explaining how and why the predictions are made. In this project, we will investigate and develop XDL approaches for gene expression based predictions that can model functional interactions and inform underlying biological functions.
Tasks
 Building DL models for gene expression based prediction of cell types, cancer stage, and drug response
 Learning and experimenting different XDL approaches
 Developing XDL for gene expression based predictions.
 Analyze the results and making improvements. Write Report
Prerequisite
 Strong background in programming, preferably in Python
 Strong background in statistics
 Excellent communication skills
 Prior experience on machine learning will be preferred
DataDriven Strategies for Largescale Cooperative Control:
Dr. Claire Walton
Project description
As autonomous vehicle platforms become more scalable in size and cost, largescale (>100) cooperative efforts have become more feasible. Largescale multiagent systems have potential applications in telecommunications, search and rescue, package delivery, and autonomous driving. Designing scalable algorithms for collision avoidance and cooperative task completion remains a challenging problem.
There are two interrelated objectives of this project. The first objective is to study the implementation of existing multiagent algorithms, such as those in [1], [2], [3] on vehicles with dynamic constraints and nonlinear dynamics. We will implement these in a simulation testbed and examine whether the stability properties of the algorithms persist. The second objective is to consider the improvement of these models, especially for nonlinear vehicle dynamics, through substituting some of their response rules with trainable neural network inserts. Training will be done on a set of formation objectives (aligning as a group to goal formations/reference trajectories) and training data will be generated through simulation.
Tasks
 Set up virtual simulation environment with either MATLAB or Python
 Demonstrate behavior of at least 1 flocking algorithm on a swarm with nonlinear dynamics
 Compare with a simple network (non deep)
 Consider analytic improvements to adjust for the vehicle dynamics
 Implement a neural network training for several of the response rules
Prerequisite
Basic knowledge of ordinary differential equations, neural networks, and either MATLAB or Python.
References:
 N. E. Leonard and E. Fiorelli, “Virtual leaders, artificial potentials and coordinated control of groups,” in Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No. 01CH37228), vol. 3. IEEE, 2001, pp. 2968–2973.
 C. W. Reynolds, Flocks, herds and schools: A distributed behavioral model.
Mehmood, N. Paoletti, D. Phan, R. Grosu, S. Lin, S. D. Stoller, A. Tiwari, J. Yang, and S. A. Smolka, “Declarative vs rulebased control for flocking dynamics,” in Proceedings of the 33rd Annual ACM Symposium on Applied Computing. ACM, 2018, pp. 816–823.
Predict realtime traffic flow and risk using AI
Project Description
The objective of this project is to enhance road safety in highrisk regions by providing realtime safety notifications to drivers using the current VIVDS. To achieve this goal, we will enable the current VIVDS to process massive realtime data, extract realtime traffic information, and identify potential safety risks using the limited computational resources from VIVDS devices. Once the highrisk threshold is reached, the framework will generate safety notifications and show the notifications on a display panel powered by energy harvesting technology before the drivers enter the highrisk regions to minimize the potential for human errors. To enhance the nighttime safety, auxiliary devices compatible with the guidelines of VIVDS will also be employed and tested.

Tasks
Task 1: Learn basic knowledge on deep learning algorithms and python programming. Weeks 12)
Task 2: Know how to access data from Crash Reporting Information System (CRIS), and Pavement Management Information System (PMIS) databases, and familiar with the image format from Video Imaging Vehicle Detection System (VIVDS). (Week 3)
Task 3: Identify highrisk regions and causes of collision using existing databases including CRIS and PMIS. (Week 46)
Task 4: Develop onboard deep learning (DL) algorithms for realtime traffic flow detection using the compatible devices deployed by VIVDS. (Week 69)
Task 5: Weekly and final reports. (Weeks 210)
Prerequisite
Basic knowledge of programing, better if know Python.