Crude scheduling is one of the largest combinatorial problems a refinery solves, and it has to be worked again every time a tanker slips, a tank frees or a unit rate changes. Exact solvers reach true optima but scale badly; manual planning is fast but conservative. A reinforcement learning policy trained on the operator's own history returns a feasible schedule in seconds, within a few percent of the mixed integer optimum, and a Pareto set the planner chooses from.
All case studies
planning horizon per schedule, across two crude units and up to eleven process units
margin gap of the learned policy against the mixed integer optimum on held out schedules
to return a full reschedule after a tanker slips or a unit rate changes
Crude scheduling is one of the largest combinatorial problems a refinery solves. Planners must sequence tanker and pipeline receipts, allocate storage and charging tanks, fix blend recipes against sulphur, acidity, density and residue limits, and set the charge to each distillation unit across a horizon of one to two weeks. The schedule has to be worked again every time a tanker slips, a tank frees, a blend target moves or a unit rate changes. Exact mixed integer formulations reach true optima but scale badly, growing from under a minute at five days to tens of hours at ten days, and any disturbance forces a full recomputation. Manual planning is fast but conservative, and leaves margin on the table.
| Training steps (millions) | RL policy |
|---|---|
| 0 | 5 |
| 0.1 | 15 |
| 0.1 | 25 |
| 0.1 | 38 |
| 0.2 | 50 |
| 0.3 | 54 |
| 0.3 | 63 |
| 0.3 | 70 |
| 0.4 | 75 |
| 0.5 | 79 |
| 0.5 | 83 |
| 0.6 | 85 |
| 0.6 | 86 |
| 0.7 | 88 |
| 0.8 | 92 |
| 0.9 | 91 |
| 1 | 94 |
| 1.2 | 95 |
| 1.4 | 95.5 |
| 1.6 | 95.5 |
| 1.8 | 95.5 |
| 2 | 95.5 |
| 2.2 | 95.5 |
| Mixed integer optimum | 100 |
EntroMetrix builds a scheduling environment on the operator's own historical data and trains a reinforcement learning policy to sequence receipts, allocate tanks, set blends and charge the units. The problem is cast as a Markov decision process: the state carries tank inventories, crude qualities and vessel arrivals, the uncertain ones held as Bayesian distributions rather than single values, the action sets receipt, transfer and charge volumes, and the reward is gross margin from a physics informed blend and yield model, net of demurrage, changeover and slop. Every action is passed through a math programming projection, so it lands on the feasible region set by tank capacities, connectivity, blend properties and unit rates before it is applied. Once trained the policy returns a schedule in seconds when a tanker slips or a unit trips, staying within a few percent of the mixed integer optimum, and surfaces a Pareto set the planner chooses from.
| Time to a schedule (s), log scale | Margin captured (% of optimum) | |
|---|---|---|
| Manual planning | 5000 | 88 |
| Mixed integer solver | 220 | 99.9 |
| RL policy | 1.5 | 95.3 |
The scheduler is built from data the plant already holds. Historical receipt logs, tank movements, blend records, unit rates and the crude assay library are enough to reconstruct the operator's own decisions and to train the policy against them. The logistics and quality limits and the operating window are set with the schedulers before any run.
On a comparable crude scheduling problem the learned policy reached within a few percent of the mixed integer optimum while returning a schedule in seconds, one to two orders of magnitude faster than a full re solve, and held that margin when a receipt was delayed because it was trained against the Bayesian distribution of arrival times and could re optimise inside the horizon rather than run a stale plan. Feasibility was guaranteed throughout by projecting every action onto the region the plant can actually operate, so proposed schedules stayed inside the accepted limits rather than breaching them as a policy trained on penalties alone would.
| Manual | RL policy | |
|---|---|---|
| Gross margin | 100 | 103.4 |
| Demurrage | 100 | 91.4 |
| CDU idle / changeover | 100 | 92.7 |
| Slop / off spec | 100 | 95.8 |
Validation is retrospective and blind. Schedules are compared against the operator's own historical schedules on held out periods before any live use, the policy is held inside logistics and process limits by a math programming projection and a constrained objective, and the scheduler runs offline for decision support, with the planner reviewing and approving the Pareto set before a schedule is issued.
| Hours into the horizon | Frozen schedule | RL policy |
|---|---|---|
| 0 | 100 | 100 |
| 4 | 99.5 | 99.5 |
| 8 | 101 | 101 |
| 12 | 98.5 | 100.5 |
| 16 | 100.5 | 100.5 |
| 20 | 100.8 | 100.8 |
| 22 | 102.5 | 102.5 |
| 24 | 100 | 96 |
| 25 | 92 | 93.5 |
| 26 | 86.5 | |
| 30 | 87 | 96.5 |
| 34 | 86.3 | |
| 38 | 87 | |
| 42 | 88 | |
| 46 | 86.5 | |
| 48 | 86 | 98.7 |
When a tanker arrives late the learned policy reschedules within seconds and recovers, while a schedule frozen before the disturbance loses margin it cannot win back without a full re solve.
The learned policy holds each distillation unit within a narrow band, while a solver chasing the cheapest paper optimum swings it and switches units to zero throughput.
A policy trained by penalty alone still breaches the accepted threshold; projecting every action onto the feasible set holds the schedule safely below it.
| Hours into the horizon | Mixed integer solver | RL policy |
|---|---|---|
| 0 | 105 | 102 |
| 2 | 90 | 108 |
| 4 | 92 | 106 |
| 6 | 80 | 110 |
| 8 | 90 | 104 |
| 9 | 58 | |
| 10 | 75 | 116 |
| 12 | 95 | 102 |
| 14 | 100 | 100 |
| 15 | 125 | |
| 16 | 0 | 102 |
| 17 | 0 | |
| 18 | 40 | 98 |
| 19 | 100 | |
| 22 | 100 | 101 |
| 24 | 110 | 103 |
| 26 | 95 | 98 |
| 28 | 90 | 99 |
| 30 | 95 | 102 |
| 32 | 124 | 103 |
| 33 | 20 | |
| 34 | 0 | 96 |
| 35 | 10 | |
| 37 | 58 | |
| 39 | 70 | |
| 41 | 80 | |
| 43 | 118 | |
| 45 | 75 | |
| 47 | 100 | |
| 48 | 95 | 104 |
| Cumulative limit violation | |
|---|---|
| Penalty trained RL | 128 |
| Projected safe RL | 16 |
A short call with our engineering team, your process data stays on site.