Projects
Traffic Congestion Control Using Reinforcement Learning:
Dr. Yufang Jin
Project description
This project aims to develop a simulation package for autonomous driving and route redirection in a designated region using reinforcement learning (RL) algorithms. The developed RL algorithms will determine the motion and routes of vehicles considering the traffic safety, shortest traveling path, shortest traveling time, and traffic conditions to reduce traffic congestion. We will further verify the algorithms using a hardware-in-the-loop testbed including scale-down tracks, car-like rovers, and traffic signaling systems. Currently, 3 rovers (75:1 scale-down) have been built, and traffic signaling systems are developed by a senior design group.
Tasks
- Learn the basic concept of the reinforcement learning algorithm; (Weeks 1-2)
- Get familiar with existing RL and interpretation algorithm developed in Python; (Weeks 2-3)
- Get familiar with Simulators including CARLA, SUMO, and Python programming (Weeks 2-3)
- Determine possible reward functions to optimize an objective function related to the motion of vehicles. (Weeks 4-6)
- Demonstrate single-vehicle motion or multiple vehicle routing with RL (Weeks 6-9)
- Write a weekly report, project progress, manuscript for paper submission, and REU final presentation; (Weeks 2-10)
Prerequisite
The basics of neural networks and Python are needed.
References:
- Introduction to Reforcement Learning by David SIlver: https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ
Magma 5G Open Source Software Technology:
Dr. Brian Kelley
GRA:
Taylor Eisman
Project Description
The goal of the project is to learn next generation 5G SW Standards, 5G Programming, and 5G Networks critical to engineers and vital for emerging software engineers, network engineers, telecomm, and advanced R&D. Learn, download, and build models of 5G Networks used in the most advanced network standard in the world. REU Students compose 5G Network models from Open Source Libraries; develop a Protocol Data Unit Session for 5G Systems. Magma is a 5G Open Source Software project develop software applications that communicate 5G data packets. Facebook has developed Magma (see https://www.magmacore.org/). It is an open source software project for 5G Telecommunication researchers, operators, and labs for rapid prototyping of 5G Core Networks. The Open Source Project provides software-centric, distributed data packet networks, containerized network function, and tools for automated network management.
Tasks
- Install open source software packages from Anaconda, Pycharm, Magma, and Github
- Review Software tutorials with your mentors, once a week Download and Open Source 5G Software, Anaconda, Magma, and Github
- Set up software platforms and review software methodology with GRA mentor
- Students Develop 5G network prototypes in Open Source Software
Prerequisite
Basic programming in Matlab/Python
Construction of Homogeneity-Inspired Neural Network:
Dr. Chunjiang Qian
Project Description
Dynamic systems use mathematical models to describe various things or environments, such as unmanned vehicles, energy-saving and SpaceX rockets. In order to simplify the modeling and control problems in dynamical systems, many big data technologies have been applied, and neural networks are one of them. Neural network has flexible architecture and can be very useful in classifications, regressions as long as enough data sets are collected. Neural networks are mainly used in the field of computer vision, but it can also be used to simulate dynamic systems and predict the future state of the system. Recent work on Neural Ordinary Differential Equations (Neural ODEs) [1] has provided a correspondence between discrete deep learning networks and continuous-time ODEs. This has given new tools for designing the network infrastructure of deep learning networks, as well as for training. [2] proposed an algebra framework and an approximation theory to reveal the relationship between the neural networks and the structure of the compositional function to be approximated. The compositional functions use directed acyclic graph (DAG) to represent a mapping between input and output and provide an intuitive structure. With this theoretical foundation, we can make error analysis of neural networks for functions as input-output relations and functions generated based on the trajectories of differential equations.
Tasks
- Use neural networks to simulate dynamic systems and predict the future state of them
- Construct a discrete-depth network by employing the homogenous functions at the nodes. The homogeneous network (Figure 1) is represented as a directed acyclic graph (DAG) and can be considered as a compositional function [2]
- Realize the new construction and training in Python/MATLAB.
- Test different constructions of new neural networks and compare the effectiveness to the traditional construction.
Prerequisite
The basics of neural networks and Python/MATLAB are needed.
References:
- T.Q. Chen, Y. Rubanova, J. Bettencourt, D.K. Duvenaud, “Neural ordinary differential equations.” Advances in neural information processing systems, 2018.
- W. Kang, and Q. Gong, “Neural Network Approximations of Compositional Functions with Applications to Dynamical Systems.” arXiv preprint, arXiv:2012.01698, 2020.
Robotic Behavior Learning From Human Demonstrations: Dr. Yongcan Cao
Project Description
Learning complex behaviors is a challenging task for robots to demonstrate human-level performance in real-world environments. The objective of the project is to learn robotic behaviors through human guidance via learning from demonstrations. One important approach to enable robotic behavior learning is through inverse reinforcement learning, which is then followed by reinforcement learning. In particular, human demonstrations will be used to learn rewards, through inverse reinforcement learning, which can be used to generate control policies.
Tasks
- Setting up simulation environments
- Generating human demonstrations
- Adapting and executing inverse reinforcement learning algorithms
- Adapting and executing reinforcement learning algorithms
- Statistical Analysis
Prerequisite
- Strong background in programming, preferably in Python
- Strong background in statistics and control systems
- Excellent communication skills
- Prior experience on machine learning and/or robotics
Optimal Charging Strategy for AI-powered EVs:
Dr. Sara Ahmed
Project Description
The number of Electric Vehicles (EV) has increased significantly in the past decades due to its advantages including emission reduction and improved energy efficiency. However, the adoption of EV could lead to overloading the grid and degrading the power quality of the distribution system. It also demands an increase in the number of EV charging stations. To meet the charging needs of 15 million EVs by the year 2030 with limited charging stations, prediction of charging needs and reallocating charging resources are in emerging needs. Our group developed three modeling methods, multi-step LSTM, one-step LSTM, and ARIMA to predict charging loads with temporal profiles for 3 charging stations in two years on the campus of the University of Texas at San Antonio.
In the summer REU program, an undergraduate student will help validate our predeveloped prediction models by developing a simulation platform and a proof of concept prototype that includes emulated smart charging systems and mini-EV vehicle prototypes that would communicate together and host our proposed management system.
Tasks
- Learn the developed prediction models; (Weeks 1-2)
- Prepare the proof-of-concept simulation/prototype; (Weeks 2-3)
- Use part of UTSA geoinformation (roadmap) as our simulation platform with entry, traffic flow information, intersection of roads, etc.
- Define number of EVs in the simulated study
- Define number and location of the EV charging stations, current utility ratio, price charging
- Develop an application that will input data to the cloud and provide the availability of the charging station; (Weeks 3-4)
- Demonstrate the proposed management system performance with computer simulations; (Weeks 4-6)
- Validate the system with the proof-of-concept prototype; (Weeks 6-9)
- Write weekly report, project progress, manuscript for paper submission, and REU final presentation; (Weeks 2-10)
Prerequisite
The basics of neural networks and Python/MATLAB are needed. Experience with App development, hardware and software experience with Arduino and Raspberry Pi.
Understanding organically organized system functions and structures using artificial intelligence approaches:
Dr. Michelle Zhang
Project Description
Currently, most systems are assumed to have a causal system structure that isolate a set of factors that can be treated as “causes” and the rest as “effects” so that various phenomena can be interpreted [1], and inferences can be made for the effective intervention of such systems. However, such an approach often fails in practice because in reality, most dynamic systems are organized organically in which, parts are mutually casual [2]. For example, within the human body, if we take the view that the heart is the root organ and it affects the lung but not vice versa, then it’s treated as a causally organized system. However, if we take the view that all major organs are affecting each other mutually, then it’s an organically organized system. There lacks development of theories and approaches for inferring components and functions of such systems. The significance of the development is immense because virtually all medically related data inferencing problems should be modeled in the proposed approach.
Tasks
- Learn the basic concept of organically organized networks and develop simulated models of such networks using Matlab; (Weeks 1-2)
- Learn the basic concept of deep learning algorithm in Python and Matlab; (Weeks 2-3)
- Apply the existing algorithm to train the deep learning model with the simulated data; (Weeks 3-5)
- Apply the model for making inference of the learned model interpretation; (Weeks 6-9)
- Write weekly report, project progress, manuscript for paper submission, and REU final presentation; (Weeks 2-10)
Prerequisite
The basics of neural networks and Python/MATLAB are needed.
References:
- Pearl, Judea. “Causal inference.” Causality: Objectives and Assessment (2010): 39-58.
- Hegel, G. W. F. (1929). Hegel’s science of logic.
Integrating on-road data for driving-safety:
Dr. Miltos Alamaniotis
Project description
On the main features that most of the autonomous cars share is modern systems for the avoidance of collisions. Though systems have greatly improved in that direction, the majority of avoidance collision systems focus on car-to-car crashes neglecting the car-to-animal collision. The objective of this project is to develop a system that predicts the behavior of animals on the fly. In that direction, deep learning with image processing will be utilized to integrate data taken from car sensors on the road. The Applied Artificial Intelligence Laboratory (directed by Dr. Alamaniotis) has novel datasets that will be used in the project. The student will have to utilize the datasets and develop the deep learning tool.
Tasks
- Data preparation (select datasets and create training and testing datasets)
- Develop of a deep network architecture
- Compare with a simple network (non deep)
- Validation of the tool
- Write Report
The student will have weekly meeting with the mentor (Dr. Alamaniotis) and will collaborate with grad students from the Applied Artificial Intelligence Laboratory.
Prerequisite
- Strong background in programming, preferably in Python/Matlab
- Knowledge of basic probabilities and statistical concepts
References:
- Munian, Y., Martinez-Molina, A., & Alamaniotis, M., “A Design and Implementation of a Nocturnal Animal Detection Intelligent System in Transportation Applications,” ASCE International Conference on Transportation & Development, June 6-9, 2021, Virtual Conference, pp. 12.
- Munian, Y., Alamaniotis, M., Martinez-Molina, A., “Intelligent System for Detection of Wild Animals Using HOG and CNN in Automobile Applications,” 9th International Conference on Information, Systems and Applications (IISA), Piraeus, Greece, July 2020, pp. 8.
Building Deep Learning Models that Understand Biology:
Dr. Yufei Huang
Project description
The ever-growing genomics data from omics profiling and single-cell technologies have spurred rapid developments of deep learning (DL) methods for genomics data analysis. While producing accurate prediction is a critical objective, the next generation omics analysis tools call for XDL models capable of explaining how and why the predictions are made. In this project, we will investigate and develop XDL approaches for gene expression based predictions that can model functional interactions and inform underlying biological functions.
Tasks
- Building DL models for gene expression based prediction of cell types, cancer stage, and drug response
- Learning and experimenting different XDL approaches
- Developing XDL for gene expression based predictions.
- Analyze the results and making improvements. Write Report
Prerequisite
- Strong background in programming, preferably in Python
- Strong background in statistics
- Excellent communication skills
- Prior experience on machine learning will be preferred
Data-Driven Strategies for Largescale Cooperative Control:
Dr. Claire Walton
Project description
As autonomous vehicle platforms become more scalable in size and cost, largescale (>100) cooperative efforts have become more feasible. Largescale multi-agent systems have potential applications in telecommunications, search and rescue, package delivery, and autonomous driving. Designing scalable algorithms for collision avoidance and cooperative task completion remains a challenging problem.
There are two inter-related objectives of this project. The first objective is to study the implementation of existing multi-agent algorithms, such as those in [1], [2], [3] on vehicles with dynamic constraints and nonlinear dynamics. We will implement these in a simulation testbed and examine whether the stability properties of the algorithms persist. The second objective is to consider the improvement of these models, especially for nonlinear vehicle dynamics, through substituting some of their response rules with trainable neural network inserts. Training will be done on a set of formation objectives (aligning as a group to goal formations/reference trajectories) and training data will be generated through simulation.
Tasks
- Set up virtual simulation environment with either MATLAB or Python
- Demonstrate behavior of at least 1 flocking algorithm on a swarm with nonlinear dynamics
- Compare with a simple network (non deep)
- Consider analytic improvements to adjust for the vehicle dynamics
- Implement a neural network training for several of the response rules
Prerequisite
Basic knowledge of ordinary differential equations, neural networks, and either MATLAB or Python.
References:
- N. E. Leonard and E. Fiorelli, “Virtual leaders, artificial potentials and coordinated control of groups,” in Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No. 01CH37228), vol. 3. IEEE, 2001, pp. 2968–2973.
- C. W. Reynolds, Flocks, herds and schools: A distributed behavioral model.
Mehmood, N. Paoletti, D. Phan, R. Grosu, S. Lin, S. D. Stoller, A. Tiwari, J. Yang, and S. A. Smolka, “Declarative vs rule-based control for flocking dynamics,” in Proceedings of the 33rd Annual ACM Symposium on Applied Computing. ACM, 2018, pp. 816–823.
Predict real-time traffic flow and risk using AI
Project Description
The objective of this project is to enhance road safety in high-risk regions by providing real-time safety notifications to drivers using the current VIVDS. To achieve this goal, we will enable the current VIVDS to process massive real-time data, extract real-time traffic information, and identify potential safety risks using the limited computational resources from VIVDS devices. Once the high-risk threshold is reached, the framework will generate safety notifications and show the notifications on a display panel powered by energy harvesting technology before the drivers enter the high-risk regions to minimize the potential for human errors. To enhance the night-time safety, auxiliary devices compatible with the guidelines of VIVDS will also be employed and tested.
-
Tasks
Task 1: Learn basic knowledge on deep learning algorithms and python programming. Weeks 1-2)
Task 2: Know how to access data from Crash Reporting Information System (CRIS), and Pavement Management Information System (PMIS) databases, and familiar with the image format from Video Imaging Vehicle Detection System (VIVDS). (Week 3)
Task 3: Identify high-risk regions and causes of collision using existing databases including CRIS and PMIS. (Week 4-6)
Task 4: Develop on-board deep learning (DL) algorithms for real-time traffic flow detection using the compatible devices deployed by VIVDS. (Week 6-9)
Task 5: Weekly and final reports. (Weeks 2-10)
Prerequisite
Basic knowledge of programing, better if know Python.
