Simulating Dynamic Vehicle Routing Problems with Athos

Complex routing problems, such as vehicle routing problems with additional constraints, are both hard to solve and hard to express in a form that is accessible to the human expert and at the same time processible by a computer system that is supposed to produce a solution of sufﬁcient quality. The formulation must be formal enough to avoid ambiguities and also comprehensible enough to be created, discussed and shared by domain experts. In this paper, we present the domain speciﬁc language Athos in which complex routing problems can be expressed in a computationally independent, human-readable form. Athos is then transformed into code that can be run in an adequate target platform. Suitable methods for solving problems are available and applied to the given problem. We present a case study in which we use a genetic algorithm to solve instances of a vehicle routing problem with time windows and demonstrate the end to end process to produce a solution in the Athos environment. Moreover, we show how the Athos system goes beyond optimisation of static routes and can be used as a tool to simulate the impact of trafﬁc and congestion on the tours. We call this extended problem a dynamic vehicle routing problem with time windows.


INTRODUCTION
Even after many years of research, routing problems are still a focus of current research efforts.Generally belonging to the class of NP-hard problems, the search for efficient heuristics remains a challenge.With logistics networks growing in size and complexity, efficiency and sustainability become central issues for the transportation industry.Rapidly changing requirements need software solutions that can be easily adapted and extended.If using general purpose languages (GPLs), changing software requires systems engineers and domain experts to interact and communicate, which often is a source of misunderstanding leading to weak models and error prone systems (Dalal and Chhillar 2012).A low level of abstraction in GPLs prevents reuse of larger building blocks and therefore implementation has to start from scratch each time a new problem has to be solved.Using a proprietary platform creates dependencies to standards imposed by a commercial software vendor that may not always be in line with the needs of the current project.We suggest using a model driven approach to overcome this dilemma.We discuss how domain experts can specify traffic related optimisation problems declaratively with our Domain-Specific Language (DSL) Athos.Athos models are accessible to humans and can be transformed into executable programs to be run in appropriate target platforms (e.g.NetLogo, Repast Symphony, or potentially any other environment).In this paper, we demonstrate Athos and its features by showing how it can be used to model both a static and a dynamised version of the Vehicle Routing Problem with Time Windows (VRPTW).In the dynamic version of the VRPTW mutual influence of traffic participants is considered.The problem runs in a network of roads defined in the model and uses a genetic algorithm to heuristically compute a solution.The models are as computationally independent as possible and therefore the algorithm and its implementation are not part of the model but of the infrastructure of the target platform.

ATHOS
Athos is a DSL designed for the domain of dynamic transportation problems.The language is implemented in Xtext (https://www.eclipse.org/Xtext/).The Athos architecture contains a generator that transforms Platform-Independent Models (PIMs) into Platform-Specific Models (PSMs); i.e. executable code to be run in a target platform.We currently support NetLogo and use it as our main development environment.Prototype support for Repast Simphony has also been implemented and tested.Code of an algorithmic nature (e.g.routing algorithms and heuristics for optimisation) is implemented in Java and made available to the target platform via libraries.The NetLogo implementation, for instance, uses extensions that can be accessed from the generated NetLogo models via the NetLogo-Extension-API.Athos deliberately does not support imperative programming as we aim to provide a purely declarative modelling language.Currently available meta-heuristics libraries in Athos are an implementation of an ACS (Ant Colony System) for solving TSP-like problems (Hoffmann, Guckert, et al. 2018) and an Evolutionary Algorithm for solving more complex vehicle routing problems like the problem presented in this paper.
The VRPTW describes the task of visiting a set of customers for delivery or pick-up of products, depending on context.Each visit must conform to time windows specified for each customer.If the vehicle does not arrive within the limits of the time window, it either has to wait until the window opens (early arrival) or the schedule is not feasible (late arrival).Visits at a customer may also consume a given amount of service time.Vehicles start and end their journeys in one of possibly many depots.If more than one depot is used, the problem is referred to as a multi-depot problem.The problem may be formulated for a fixed number of vehicles or the number of vehicles may be part of the objective function of the optimisation problem.VRPTWs can be used to optimise a single objective (e.g.overall distance travelled) or multiple objectives (e.g.number of vehicles and distance travelled) (Dabia, Demir, and Woensel, van 2014).The VRPTW is highly relevant to real-world problems both in an operational (N.B. Urquhart, Hart, and Judson 2015) and a planing context (N.Urquhart and Fonzone 2017).The user of the VRPTW-solving software will be a domain expert, e.g. a logistics analyst or transport planner.Many industrial contexts will include additional specific constraints defined by the business context, i.e. working conditions of staff, types of vehicle in use, environmental or financial considerations.The implementation of such additional, possibly very versatile, constraints leads to an increased workload for professional programmers using a conventional GPL.A DSL as Athos will potentially reduce development time and give domain experts a tool to easily modify and extend a model without having to access software developers.Obviously, any problem instance of a VRPTW requires travel time data between customers and depots.Depending on the optimisation criterion used, it may also be necessary to retain distance or emissions data as well.Such data can be held in an origin-destination matrix (see e.g.Dantzig, Ramser, and Hubert 1959), or simply be calculated using the Euclidean distance between customers.However, real-world examples (N.Urquhart and Fonzone 2017; N. B. Urquhart, Hart, and Judson 2015) usually use an underlying street graph and apply pathfinding algorithms to find routes between customers.Summarising the discussion, we see that a DSL supporting the domain of vehicle routing must be capable of expressing the many different formulations of a VRP including differing vehicle types, time windows, service times, capacity constraints and driving time constraints.At the same time, it must also support the modelling of the underlying street graph.The following step by step example illustrates the modelling features of the language and how a VRPTW can be described.The product to be delivered is soap each item having a weight of 1 weight unit.The model contains a single agent type named delivery with two states awt and die.The agents of this type wait for an optimal tour to be computed and then start delivering goods according to the tour received thereby satisfying demands of the customers on that tour.Travel time will be computed using the default duration function that uses length and speed of the agents and counts in the defined congestion factors so that a high amount of traffic in the network has an impact on the delivery.Behavioural patterns of agents are defined by means of behaviour blocks.For each agent type a single behaviour block with an arbitrary number of behaviour states can be defined.Behaviour states correspond to the states of an implicit finite automaton.A state consists of a perceivable action and a specification of transitions.Events can be defined as stimuli that trigger a transition and entail a change of state.The Athos meta-model reflects all of the elements of the language.Figure 1 shows how agent states are represented there.AgentTypes are linked to exactly one AgentBehaviourBlock, which contains one or more AgentBehaviourStates.The state of an agent corresponds to exactly one observable behaviour that the agent exhibits when being in the respective state.This observable behaviour is an instance of AgentBehaviourDescription. Athos has a set of built-in AgentBehaviourDescriptions which will continuously be extended in future versions.Additionally, an AgentBehaviourState contains an arbitrary number of AgentBehaviourTransitions, which trigger a change to a target state depending on a condition.Possible target states are other named states in the AgentBehaviourBlock (in Figure 1 or anonymous states defined in the AgentBehaviourTransition.The network is a complete graph with edge lengths equal to the Euclidean distance between the respective end nodes.Nodes are defined with their coordinates.As the graph is defined to be complete, the list of edges is empty.The length of the edges is used as travel time.Note that this is only an academic example -any other duration function could be defined and used here.Sources and demands are defined using nodes of the network.The keyword ea indicates that an evolutionary algorithm is to be used to compute the tours for the vehicles.Additional parameters for the algorithm are provided.For each demand quantities, time windows, and service time are defined.Agents of a given type can be monitored by using metrics in which indicators can be defined that can either accumulate values or set values.These indicators are collected for each agent and can be viewed for each single agent or condensed into overall statistics.As any other feature of Athos, metrics have a representation in the meta model analogous to that of the behaviours.However, for the sake of brevity this is not discussed here.Beyond the optimisation of tours with the built in evolutionary algorithm, Athos can run simulations of dynamic delivery scenarios with these optimised schedules in which the effect of traffic and congestion in the network can be measured up with the individually defined metrics.While a static VRPTW optimises tours once according to a defined objective function, a dynamic VRPTW goes beyond that and is sensitive to dynamic aspects of traffic in the underlying network.We discuss and compare a static and a dynamic VRPTW in the case study presented in the next section.

CASE STUDY
In this section, we will present a case study that compares the static and dynamic variants of the VRPTW problem.We will analyse how a changed traffic situation influences the success of the planned tours.First we will use Athos to define a static VRPTW.In addition, we will define some metrics to see how the vehicles perform when the traffic situation stays exactly the same throughout the entire simulation.In a second step, we will transform the VRPTW into a dynamic VRPTW by adding additional noise-agents that will induce congestion effects inside the network.We will use the defined metrics to see how congestion influenced the outcome of the calculated VRPTW solution.Note that the complete Athos program, the generated NetLogo programs (together with the required extension) as well as some videos showing the simulation can be obtained from https://athos.mnd.thm.de/public/ecmscasestudy.html.
Figure 2 illustrates the graph used in this case study.The graph is an artificially simplified version of an urban area with the following characteristics: • At its core, the area features a beltway.The roads here are highly dependent on each other so that congestion on one road directly expands to other roads of the beltway.• The depot is located at the very centre of this beltway.
Access roads to the depot also belong to the beltway and thus are also affected by any congestion on the beltway.• The centre of this area also features a network of highways with high capacity.These highways are independent so that congestion on one highway does not have ripple effects on adjacent highways.• Suburban areas are accessible through roads of less capacity.These roads are more susceptible to congestion when used by a queue of cars or cars with high congestion factors like heavy-goods vehicles.The first two lines of the listing above give a name to the model and define its global boundaries.The definition of the agent types used in the case study is omitted in the listing but will be presented shortly.Lines 4 to 7 contain the functions that are associated to the two different road types used in the case study.The function associated to edges which represent highways was aptly named "highway".The meaning of this function is that the length of a highway defines the minimum amount of time a vehicle needs to cross it.For example, a highway of length 200 would require 200 ticks to be crossed by a vehicle, given that the vehicle has a congestion factor of zero and the summed up congestion factor of all other vehicles on the road is also zero.This is due to the fact that the congestion factor of each vehicle on a road additionally increases the time required to cross the respective road.
Roads that belong to the same path share their accumulated congestion factor.In the listing, edge e01 and e12 belong to the same path.To account for the fact that normal roads take longer to travel and are more susceptible to congestion than highways, the road function multiplies the length of the road by three and the accumulated congestion factors by four.The above listing shows the specification of the homogeneous fleet of vehicles located at the depot.Lines 3 to 6 specify that these vehicles have a congestion factor of zero and thus do not congest the roads.Each vehicle can provide a customer with 180 units of a given product.Line 4 specifies that this type of vehicle waits at a depot until it is assigned a tour.As the metrics that we will define in the next step do only apply for existing agents, lines 5 and 6 tell a vehicle to idle at the depot for 1,000 ticks before leaving the simulation.In line 9, node n0 is defined to be a depot.Customers are defined inside the brackets.Note that the code is intention-ally no longer computationally independent, because the ea keyword explicitly specifies the application of an evolutionary algorithm.This allows to specify the parameters used in the respective algorithm -in this case, the population is set to a size of 30.The depot searches for a set of tours for its vehicles at the very beginning of the simulation.This is specified in line 11 with the keyword at followed by the value zero.
The depot uses a genetic algorithm (Hoffmann, Chalmers, et al. 2019) that internally builds a complete graph of all customer nodes (Hoffmann, Guckert, et al. 2018) and optimises the function given in (1).Here, K denotes the set of vehicles at the depot, N the set of nodes of the complete graph (including the depot), t ij the travel time from node i to node j and x k ij a decision variable that is set to 1 if vehicle k travels from node i to node j and 0 otherwise.A parameter value of 0.001 weights the total distance covered by all vehicles.Analogously, 1000 is the weight for the number of vehicles used.The complete set of parameter values is given in Table II.Finally, lines 12 to 17 contain the demand specifications.Note that the depot appears here in order to define the latest point in time for the return of the vehicles.Table I summarises the constraints related to the customers in this case study.The Athos code presented so far allows us to run the simulation in a "noise-free" mode: The depot calculates a solution for the VRPTW and sends out the vehicles to serve the customers.In order to transform the static VRPTW into a dynamic version, we will run a second batch of simulations in which additional noise-agents will increase travel times on the city ring and some of the roads/highways in the network.To this end, the above listing introduces another agent type with a congestion factor of 150.Due to its high congestion factor, this type of agent considerably slows down traffic on its current road (or path of roads).The agent follows a predefined route (specified in lines 3 and 4) 5 times and then disappears.Lines 9 to 11 define the nodes and the exact point in time where an instance of this type of agent appears.Note that even though node n23 is not part of the specified route of nodes for that type of agent, it can still appear at this node.The agent then uses the fastest way to the first node of the tour specified for this agent type (n1).The final part of the simulation specification for this case study is the definition of a set of metrics.In this case study, we are interested in the cumulative distance travelled by the delivery vehicles.Moreover, the exact time vehicles had to wait due to an early arrival at a customer might give some insight on the efficiency of the calculated tour.Also, it might be important to now the accumulated time by which vehicles arrived to late at a customer and the total number of time windows violated and time windows met.These metrics are specified in the following listing.
1 << as before >> 2 defineMetrics updateRate 10 Table II and Table III summarise the results of ten simulation runs each for the problem without noise agents and the problem with noise agents that introduce dynamism through reduced travel speeds.As was to be expected, in the simulations without noise-agents, no time windows were violated.In addition to the 16 customers defined for the problem, the metric also counts the timely return of a vehicle to the depot as a met time window.Since the evolutionary algorithm used to solve this problem is not deterministic, some runs feature solutions with three and some with four vehicles resulting in 19 or 20 met time windows.The introduction of noise-agents changes the situation dramatically.The noise agents effectuate the movement speed on their respective roads in a way that the delivery vehicles do no longer meet all time windows.In fact, nearly half of the defined time windows are violated.In each of the ten simulations with noise-agents the accumulated ticks by which time windows were missed is around 760.
In both cases the distance travelled by the vehicles was nearly which was to be expected.The slower movement on the roads caused the vehicles to arrive at their customers later than originally calculated which is also reflected in the amount of ticks that vehicles arrived too early at their customers which is reduced by around 6.7 ticks.
In our future work, we will use Athos to further research into dynamic VRPTW to provide strategies that provide satisfactory solutions even in case of sudden traffic surges.Steil et al. (Steil et al. 2011) discuss an approach that encompasses all aspects involved in the domain of patrol routing algorithms.It covers all stages in the development of patrol routes from the specification (expression) and simulation-based assessment (execution and evaluation) to the translation of patrol routes to the real-world (engagement).Accordingly, they call their approach the 4Es approach.The 4Es approach is similar to the approach presented in this paper in that it integrates the expression, simulation and evaluation in an appropriate environment.Moreover, it also uses a DSL for the expression of routing algorithms.Their DSL, called Turn, allows to define the next destination of an agent in a road network by means of set reduce functions that can be chained to successively reduce the set of all nodes until only one node is left which is then selected as an agent's next destination.This way the agents in the simulation follow a pre-defined routing strategy.The system evaluates routing strategies by application of four distinct metrics.These metrics provide information on the time it took a first-responding agent to get to the node where an event occurred, the percentage of  Agents move along the underlying graph among neighbouring nodes one node per time step.Due to the fact that there are no congestion effects or any changes in the movement speeds of agents, their approach cannot be used to simulate dynamic vehicle routing problems where travel times are subject to fluctuations depending on the current traffic situation.

WORK
Another platform for traffic and transport simulations is MAT-Sim (Horni, Nagel, and Axhausen 2016).MATSim is a mutliagent microsimulation system based on the co-evolutionary principle.This means that the agents in the system are equipped with a set of plans to follow.Throughout multiple iterations the agents try and evaluate the outcome of the plans at their disposal.In each cycle a plan is selected, applied and evaluated.With a given probability, agents modify different dimensions of their plans.For example, agents can vary the time they leave a given location, choose a different route or switch to a different mode of transport.Each agent seeks to optimise its individual outcome.
Macijewski and Nagel present a MATSim-based approach to evaluate algorithms for the DVRP (Maciejewski and Nagel 2012).At the same time, their work intends to plan demandresponsive transport services (DRT severcies) using the MAT-Sim framework.In their approach MATSim is used to calculate time-dependant travel times for a given scenario.The calculated data is then merged with additional demand and supply data which results in a (dynamic) VRPTW.This way, the authors designed four scenarios.The scenarios were designed to closely resemble traffic situations common to urban environments.Two of the scenarios analysed courier services while the other two scenarios investigated taxi services.They calculated solutions in two ways: First, they solved the problem using time-dependant travel times.Second, they produced solutions based on average travel times.The latter results were then applied to the problem with time-dependant travel times.The authors found that for the couriers services the routes calculated on the basis of the time dependent travel times considerably outperformed those based on average travel times.Their explanation is that knowledge of time-dependant travel times allows for the calculation of routes that avoid congested roads.Interestingly, for the taxi services travel times for the solutions of both approaches were rather similar which the authors explain with the nature of the demands to taxiservices which does not lend itself to careful planning.For both scenarios, the solutions based on average times violated the defined time windows when applied to the problem that featured time-dependent travel times.GAMA (Grignard et al. 2013) is a sophisticated simulationplatform.It features the GAML DSL which was designed for modelling agent-based simulations.The DSL's meta-model elements can be divided into elements for the definition of aspects related to entities, space and time.A species element is to GAML what a class is to object-oriented languages like Java.A distinct feature of the GAML meta-model is that it allows agents to form containment hierarchies that can be used to model different levels of detail in a simulation.The species definition is also used to equip agents with skills like movement and attributes like movement speed.The species of an agent also defines a set of actions and reflexes.Actions represent behaviour that an agent executes when asked to.By contrast, a reflex represents behaviour that the agent executes in every step of the simulation given that all guard conditions hold.Even though GAML could also be used to model transport and routing problems, it requires modelling in a language not specifically tailored towards this domain.Thus, models are on a less abstract level which makes harder to comprehend and communicate by domain experts.

CONCLUSIONS AND FUTURE WORK
We have demonstrated how Athos can be used to model dynamic vehicle routing problems and how solutions can be computed.Besides the evolutionary algorithm presented here Athos provides other heuristics for solving a variety of complex routing problems (see Hoffmann, Guckert, et al. 2018 andHoffmann, Chalmers, et al. 2019).
At the moment, we extend the Athos environment with a flexible interface to Open Street Map (www.openstreetmap.org)so that the definition of the underlying network can be generated from OSM data rather than be coded manually.Beyond that our concern is to measure general usability aspects of the language by letting domain experts assess the applicability of Athos.While we currently aim at improving the modeling capabilities of Athos our long-term intention is to develop an integrated instrument that allows a domain expert to describe and solve real world traffic related routing problems without any need for algorithmic decisions.The system will choose appropriate methods and heuristics and generate efficient bestpractice code for the target platform.

Fig. 2 .
Fig. 2. Artificial graph used in the case study.

TABLE I .
CONSTRAINTS OF THE VRPTW.

TABLE II .
RESULTS FOR TEN RUNS OF A NOISE-FREE SIMULATION (PARAMETER SETTINGS: POPSIZE = 30; SIMPLEPERMUTATIONPROB = 0.9; MAXDISTANCE = 4; GENERATIONS = 80; WNOOFTOURS = 100; WTOTALDIST = 0.001; TRMTSIZE = 4; TAKEBESTPROB = 0.8; MUTATIONPROB = 0.1. TABLEII).network visited by agents per day, the number of nodes visited that were in a state in which an event of interest is likely to occur (so-called hot nodes), an the amount of time agents spent at such hot nodes.Despite the mentioned similarities, the work of Steil et al. differs considerably from our work when looked at in more detail.First of all, the two research efforts target two different domains.As is pointed out by the authors, patrol routing problems share some features with vehicle-routing problems but greatly differ in what practitioners ultimately aim to achieve.In VRPs, in the majority of cases, the objective is the minimisation of a given cost function.By contrast, patrol routing dispatchers often do not search for a solution that optimises any specific value.Instead they search for routes that bring about satisfactory values for certain metrics as long as the routes followed by the agents are somewhat unpredictable and are non-deterministic.In contrast to the Turn DSL, Athos allows an explicit definition of a list of nodes to visit in the exact specified order or the definition of a set of nodes which have to be visited in an order that optimises a user-defined function.Most importantly, in the simulation framework of Steil et al., there is no concept of velocity or congestion.