EV2Gym-Driveway: Semi-Centralized EV Charging with Reinforcement Learning

TLDR

• Built a realistic EV travel dataset from the NHTS to model household driving patterns across the U.S.
• Developed EV2Gym-Driveway, a custom OpenAI Gymnasium simulator modeling neighborhoods where each home owns one EV.
• Trained PPO and DQN agents to coordinate charging and discharging, reducing peak grid load and cost compared to a naive “charge immediately” baseline — with PPO achieving a 12.5% cost reduction when coordinating 3 vehicles.

Project Overview

As electric vehicles (EVs) become more common, their potential as distributed energy storage is gaining attention. This project explores how EVs parked in home driveways can be coordinated using reinforcement learning to lower electricity costs and reduce stress on the grid.

We built a custom simulation environment — EV2Gym-Driveway — to model a realistic neighborhood setting where each EV is assigned a daily travel pattern. Our goal was to develop a reinforcement learning system that intelligently schedules charging and discharging under real-world constraints.

We trained both a Proximal Policy Optimization (PPO) agent and a Deep Q-Network (DQN), comparing them against a naive Charge As Fast As Possible (CAFAP) baseline. PPO and DQN both succeeded at reducing peak grid load and cost — but primarily at small coordination group sizes. As group size increased, performance became unstable, highlighting the challenges of reward shaping, hyperparameter tuning, and scalable coordination under semi-centralized control.

Full Report and Code

Results: Reducing Charging Cost

What I Learned