Widespread adoption of autonomous vehicles will not become a reality until solutions are developed that enable these intelligent agents to co-exist with humans. This includes safely and efficiently interacting with human-driven vehicles, especially in both conflictive and competitive scenarios. We build up on the prior work on socially-aware navigation and borrow the concept of social value orientation from psychology —that formalizes how much importance a person allocates to the welfare of others— in order to induce altruistic behavior in autonomous driving. In contrast with existing works that explicitly model the behavior of human drivers and rely on their expected response to create opportunities for cooperation, our Sympathetic Cooperative Driving (SymCoDrive) paradigm trains altruistic agents that realize safe and smooth traffic flow in competitive driving scenarios only from experiential learning and without any explicit coordination. We demonstrate a significant improvement in both safety and traffic-level metrics as a result of this altruistic behavior and importantly conclude that the level of altruism in agents requires proper tuning as agents that are too altruistic also lead to sub-optimal traffic flow.
Seamless and safe highway merging requires all AVs working together and accounting for the human-driven vehicles’ utility.(a) Egoistic AVs solely optimize for their own utility
(b)Altruistic AVs compromise on their welfare to account for the human-driven vehicles
Cooperative Autonomous Vehicles that Sympathize with Human Drivers
Behrad Toghi, Rodolfo Valiente, Dorsa Sadigh, Ramtin Pedarsani, Yaser P. Fallah
Multi-channel VelocityMap state representation embeds the speed of the vehicle in pixel values.
Our deep Q-network with 3D Convolutional Architecture
The multi-agent training and policy dissemination process
Human-driven Merging vehicle
HV+E .Egoistic autonomous vehicles solely optimize for their own utility
HV+SC . Cooperative Sympathetic autonomous vehicles account for the human-driven vehicles’ utility
Autonomous Merging vehicle
AV+E . .Egoistic autonomous vehicles. Although an autonomous merging vehicle acts more aggressively, it still fails to merge safely when the other AVs are egoistic
AV+SC .Cooperative Sympathetic autonomous vehicles account for the all vehicles’ utility
How cooperative sympathetic autonomous vehicles act in more complex mixed-autonomy multi-lane highways?
Weakly Sympathetic AVs: tries to open-up space for the merging vehicle but still prioritize its own utility and speed up.
Strongly Sympathetic AVs: compromises on its own utility to ensure safe merging for the merging vehicle.
Comparison between egoistic, cooperative-only, and sympathetic cooperative autonomous agents and how they interact with an autonomous (top) or human-driven (bottom) mission vehicle. A set of sampled mission vehicle’s trajectories are illustrated on the left-side, relating to each of the 6 experiments.
Performance comparison of related architectures. Our Conv3D architecture outperformed the others as the level of randomness increases and agents face episodes different than what they had seen during the training.
A set of sample trajectories of the merging vehicle shows mostly successful merging attempts in HV+SC, compared to the failed attempts in HV+E
Comparing weakly and strongly sympathetic autonomous agents: (left) Speed profiles of the “guide AV” and (right) Sample snapshots.
Training performance of the three benchmark network architectures.
Tuning SVO for autonomous agents reveals that an optimal point between caring about others and being selfish exists that eventually benefits all the vehicles in the group.