Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a582aa3
refactor(scrapers): move the German EV sweep into BaseScraper
yellowhat Oct 23, 2026
e7ae3a9
feat(db): store EV fees and upstream timestamps on prices
yellowhat Oct 23, 2026
7934e00
feat(api): return EV prices and fees from the station endpoints
yellowhat Oct 23, 2026
6a00183
feat(popup): show EV prices and charging fees
yellowhat Oct 23, 2026
c4362fa
feat(scrapers): add OCPDB as an opt-in German EV source
yellowhat Oct 23, 2026
0f7011e
feat(scrapers): make OCPDB the default German EV source
yellowhat Oct 23, 2026
573cc05
feat(planner): price EV plans with kWh and session fee
yellowhat Oct 23, 2026
319be97
docs: OCPDB as Germany's default EV source, with prices
yellowhat Oct 23, 2026
e56675a
docs: stop calling every charger unpriced
yellowhat Oct 23, 2026
f7262d8
fix(planner): weigh stop fees when pruning candidates
yellowhat Oct 23, 2026
62a6cf1
fix(ocpdb): read VAT given as a fraction or as zero
yellowhat Oct 23, 2026
9af1c52
fix(ocpdb): price a pin from the EVSEs at its top power
yellowhat Oct 23, 2026
e13889f
fix(planner): keep unpriced chargers in priced EV plans
yellowhat Oct 23, 2026
2b8535d
fix(ocpdb): merge register sites only at 5 dp
yellowhat Oct 23, 2026
0793e19
fix(ocpdb): drop implausible per-minute fees and show tiny ones per hour
yellowhat Oct 23, 2026
f4a5f20
perf(ocpdb): parse location pages as they arrive
yellowhat Oct 23, 2026
3dc42bb
fix(ocpdb): leave tariffs with conflicting energy prices unpriced
yellowhat Oct 23, 2026
7c35f2c
docs: credit OCPDB under dl-de/by-2-0 and note the rollback cleanup
yellowhat Oct 23, 2026
6e9a9c4
fix(planner): plan EV by time when the filters leave no priced charger
yellowhat Oct 23, 2026
9c90da4
fix(ocpdb): fail the run on unparseable records instead of dropping them
yellowhat Oct 23, 2026
1493430
fix(planner): cost unpriced chargers at the median price, not as free
GeiserX Oct 23, 2026
8327167
fix(ocpdb): skip the odd bad record, cap per-minute fees at 0.30, anc…
GeiserX Oct 23, 2026
25cc4c5
fix(ocpdb): never show an AC rate on a DC pin, and follow OCPI's firs…
GeiserX Oct 23, 2026
9d8a585
fix(planner): leave chargers with an unknown detour out of the median…
GeiserX Oct 23, 2026
16566c7
docs: give the app 1.5 GB in the sample compose
GeiserX Oct 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -90,13 +90,22 @@ PUMPERLY_DEFAULT_COUNTRY=ES
# Set to 0 to disable EV charger scraping (default: enabled when OCM key is set)
# PUMPERLY_EV_ENABLED=1

# Germany EV chargers: which of the two sources supplies them. Defaults to the
# BNetzA Ladesäulenregister — the official register every operator of a public
# charge point must file into, ~74,000 locations, no API key and no signup.
# Set to "ocm" to keep OpenChargeMap instead. The two never run together: the
# BNetzA scraper retires OpenChargeMap's German rows on its first healthy run,
# and running both would double-pin the country.
# PUMPERLY_DE_EV_SOURCE=bnetza
# Germany EV chargers: which source supplies them. Defaults to "ocpdb", the Open
# ChargePoint Database via MobiData BW: the BNetzA register merged with the
# operators' own feeds, plus ad-hoc €/kWh prices. No API key, no signup.
# "bnetza" uses the BNetzA Ladesäulenregister alone (no prices), "ocm" keeps
# OpenChargeMap. Only one ever runs. ocpdb and bnetza retire the other
# sources' German rows on their first healthy run; ocm does not, so switching
# to it leaves the `ocpdb-`/`bnetza-` rows behind to delete by hand.
# PUMPERLY_DE_EV_SOURCE=ocpdb

# Stations a run must refresh before the OCPDB scraper is allowed to prune
# stale rows or retire the BNetzA/OpenChargeMap German ones.
# PUMPERLY_OCPDB_MIN_STATIONS=10000

# Days an OCPDB operator feed may go without refreshing before its prices are
# ignored (its chargers then show unpriced). Failed feeds are always ignored.
# PUMPERLY_OCPDB_MAX_FEED_AGE_DAYS=3

# Stations a run must refresh before the BNetzA scraper is allowed to prune
# stale rows or retire OpenChargeMap's German ones. A safety floor: a degraded
Expand Down
6 changes: 3 additions & 3 deletions docs/configuration/api-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ This page lists every API key Pumperly can use: where to get it, what it unlocks
|---|---|---|---|
| [`TANKERKOENIG_API_KEY`](#tankerkonig-germany) | Tankerkönig | Germany's fuel prices | No German fuel prices. Each run fails and the last prices stay. |
| [`FUELPRICES_DK_API_KEY`](#fuelpricesdk-denmark) | fuelprices.dk | Denmark's fuel prices | A fallback feed is tried, and it no longer answers. |
| [`PUMPERLY_OCM_API_KEY`](#open-charge-map) | Open Charge Map | EV chargers in every country without an official registry | No chargers from Open Charge Map. Germany still gets chargers from BNetzA. |
| [`PUMPERLY_OCM_API_KEY`](#open-charge-map) | Open Charge Map | EV chargers in every country without an official registry | No chargers from Open Charge Map. Germany still gets chargers from OCPDB. |
| [`PUMPERLY_REVE_API_KEY`](#mapa-reve-spain) | Mapa REVE | Spain's chargers from the official registry | Spain's chargers come from Open Charge Map instead. |
| [`NSW_FUEL_API_KEY`](#nsw-fuelcheck-australia) and [`NSW_FUEL_API_SECRET`](#nsw-fuelcheck-australia) | NSW FuelCheck | Fuel prices in New South Wales, Australia | Only Western Australia updates. |

Every other source, in every other country, is keyless. That includes Germany's charger registry, the [BNetzA Ladesäulenregister](../data/ev-sources.md). [Fuel price sources](../data/fuel-sources.md) and [EV charging sources](../data/ev-sources.md) list them all.
Every other source, in every other country, is keyless. That includes Germany's charger sources, the [Open ChargePoint Database](../data/ev-sources.md#ocpdb) and the [BNetzA Ladesäulenregister](../data/ev-sources.md#bnetza). [Fuel price sources](../data/fuel-sources.md) and [EV charging sources](../data/ev-sources.md) list them all.

## Setting a key {#setting-a-key}

Expand Down Expand Up @@ -107,7 +107,7 @@ Keys are read when the app starts. Restart it after adding or changing one.
- **Get a key:** create a free account at [openchargemap.org](https://openchargemap.org) and create an API key there.
- **Variable:** [`PUMPERLY_OCM_API_KEY`](../reference/environment-variables.md#pumperly_ocm_api_key).
- **How it is used:** the key goes in the `X-API-Key` header. Each `EV_<CC>` scraper runs once a day by default. A small country takes one request. The API returns at most 5,000 chargers per request, so for a large country the scraper splits the map into smaller boxes. By default it sends one request at a time across all countries, with 600 ms between requests, and at most 800 requests per country per run. [Open Charge Map tuning](../reference/environment-variables.md#open-charge-map-tuning) lists the settings.
- **Without it:** every Open Charge Map scraper skips its run and writes nothing. Chargers already in the database stay. Germany still gets chargers from the keyless BNetzA register, and Spain from REVE if you set that key. The log shows, for each country:
- **Without it:** every Open Charge Map scraper skips its run and writes nothing. Chargers already in the database stay. Germany still gets chargers from the keyless OCPDB, and Spain from REVE if you set that key. The log shows, for each country:

```text
[ocm] PUMPERLY_OCM_API_KEY not set, skipping
Expand Down
25 changes: 14 additions & 11 deletions docs/configuration/countries-and-schedule.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ Each scheduled scraper has a key. You use keys in [`PUMPERLY_ENABLED_COUNTRIES`]
| `EV_<CC>` | EV chargers for one country from Open Charge Map | `EV_FR`, `EV_US` |
| `EV_ES_REVE` | Spain's chargers from Mapa REVE, the official registry | `EV_ES_REVE` |
| `EV_DE_BNETZA` | Germany's chargers from the BNetzA Ladesäulenregister | `EV_DE_BNETZA` |
| `EV_DE_OCPDB` | Germany's chargers and their prices from the Open ChargePoint Database | `EV_DE_OCPDB` |
| `STATIC_<SOURCE>` | A community dataset committed to the repository | none ship today |

`<CC>` is an [ISO 3166-1 alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) country code. The 39 fuel-price countries are listed in [the country table](#default-country-and-fuel). The United States has chargers only, under `EV_US`, because it has no national fuel price source.
Expand Down Expand Up @@ -68,7 +69,7 @@ With [`PUMPERLY_EV_ENABLED=0`](../reference/environment-variables.md#pumperly_ev
| `ES,PT,FR` | Fuel prices and chargers for the three countries |
| `AU,AU_NSW` | Fuel prices for Western Australia and New South Wales, and Australia's chargers |
| `US` | Chargers in the United States only |
| `EV_DE` | Germany's chargers only, from the BNetzA register by default |
| `EV_DE` | Germany's chargers only, from OCPDB by default |
| `ES,XX` | Same as `ES`. `XX` is ignored. |

The value also sets the `enabledCountries` list that [`/api/config`](../reference/api.md) and `/api/stats` report. That list only holds the 39 fuel-price country codes, so `US`, `AU_NSW` and `EV_` entries never appear in it.
Expand All @@ -86,32 +87,34 @@ Some sources are blocked or paused upstream, so a country can be enabled and sti

## Spain and Germany: one EV source each {#spain-and-germany-ev-sources}

Spain and Germany each have two possible charger sources: Open Charge Map, which is crowdsourced, and an official registry. The two overlap heavily, so running both would put two pins on most chargers. The scheduler always keeps exactly one per country. Listing both keys does not change that.
Spain and Germany each have more than one possible charger source: Open Charge Map, which is crowdsourced, and an official one. Germany has two official ones, the BNetzA register and OCPDB, which contains that register. The sources overlap heavily, so running two would put two pins on most chargers. The scheduler always keeps exactly one per country. Listing several keys does not change that.

| Country | Official registry | Picked when | Otherwise |
| Country | Official source | Picked when | Otherwise |
|---|---|---|---|
| Spain | Mapa REVE (`EV_ES_REVE`) | [`PUMPERLY_REVE_API_KEY`](../reference/environment-variables.md#pumperly_reve_api_key) is set | Open Charge Map (`EV_ES`) |
| Germany | BNetzA Ladesäulenregister (`EV_DE_BNETZA`) | Always, unless the next column applies | Open Charge Map (`EV_DE`) when [`PUMPERLY_DE_EV_SOURCE=ocm`](../reference/environment-variables.md#pumperly_de_ev_source) |
| Germany | Open ChargePoint Database (`EV_DE_OCPDB`) | Always, unless the next column applies | The BNetzA register (`EV_DE_BNETZA`) when [`PUMPERLY_DE_EV_SOURCE=bnetza`](../reference/environment-variables.md#pumperly_de_ev_source), Open Charge Map (`EV_DE`) when it is `ocm` |

The log shows the choice at start-up:

```text
[scraper] Spain EV: using Mapa REVE (official registry) instead of OpenChargeMap
[scraper] Germany EV: using the BNetzA Ladesäulenregister instead of OpenChargeMap
[scraper] Germany EV: using OCPDB (BNetzA register + operator feeds, with prices)
```

Each registry deletes Open Charge Map's rows for its country once it has taken over:
Each official source deletes the other sources' rows for its country once it has taken over:

- **BNetzA** arrives as one daily file. Its first healthy run replaces the whole country and deletes the Open Charge Map rows. "Healthy" means it refreshed at least [`PUMPERLY_BNETZA_MIN_STATIONS`](../reference/environment-variables.md#pumperly_bnetza_min_stations) stations.
- **OCPDB** is read in full every 6 hours. Its first healthy run replaces the whole country and deletes the Open Charge Map and BNetzA rows. "Healthy" means it refreshed at least [`PUMPERLY_OCPDB_MIN_STATIONS`](../reference/environment-variables.md#pumperly_ocpdb_min_stations) stations.
- **BNetzA** arrives as one daily file. Its first healthy run replaces the whole country and deletes the Open Charge Map and OCPDB rows. "Healthy" means it refreshed at least [`PUMPERLY_BNETZA_MIN_STATIONS`](../reference/environment-variables.md#pumperly_bnetza_min_stations) stations.
- **REVE** fills in over many hourly runs. Open Charge Map's Spanish rows stay on the map until REVE holds [`PUMPERLY_REVE_CUTOVER_RATIO`](../reference/environment-variables.md#pumperly_reve_cutover_ratio) of the registry, 95% by default. Until then, expect two pins on many Spanish chargers. [API keys](api-keys.md#mapa-reve-spain) has the timing.

!!! warning "Switching back leaves the registry's rows behind"
Nothing deletes a registry's rows if you switch back to Open Charge Map. Open Charge Map then adds its own, and most chargers get two pins. After switching Germany back to `ocm`, or removing the REVE key, delete the registry rows yourself:
!!! warning "Switching back leaves the official rows behind"
Open Charge Map deletes nothing when you switch back to it. It adds its own rows, and most chargers get two pins. After switching Germany to `ocm`, or removing the REVE key, delete the official rows yourself. Switching between `ocpdb` and `bnetza` needs nothing: each deletes the other's rows.

```sql
-- Germany, after PUMPERLY_DE_EV_SOURCE=ocm
DELETE FROM stations
WHERE country = 'DE' AND station_type = 'ev_charger' AND external_id LIKE 'bnetza-%';
WHERE country = 'DE' AND station_type = 'ev_charger'
AND (external_id LIKE 'ocpdb-%' OR external_id LIKE 'bnetza-%');

-- Spain, after removing PUMPERLY_REVE_API_KEY
DELETE FROM stations
Expand All @@ -134,7 +137,7 @@ The defaults follow how often each source changes:
| 1 hour | `FR`, `DE`, `EV_ES_REVE` |
| 2 hours | `AT` |
| 4 hours | `GB` |
| 6 hours | `SI`, `NL`, `BE`, `DK`, `NO`, `IS`, `CY` |
| 6 hours | `SI`, `NL`, `BE`, `DK`, `NO`, `IS`, `CY`, `EV_DE_OCPDB` |
| 12 hours | every other fuel key, including `AU_NSW` |
| 24 hours | every `EV_<CC>` key and `EV_DE_BNETZA` |

Expand Down
15 changes: 8 additions & 7 deletions docs/data/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,29 +102,30 @@ EV chargers come from three sources. Every fuel country above also gets EV charg
|---|---|---|---|---|
| Every supported country and the United States | `EV_<code>`, for example `EV_FR` | [Open Charge Map](ev-sources.md#open-charge-map) | 24 h | `PUMPERLY_OCM_API_KEY` |
| Spain | `EV_ES_REVE` | [Mapa REVE](ev-sources.md#mapa-reve) | 1 h | `PUMPERLY_REVE_API_KEY` |
| Germany | `EV_DE_BNETZA` | [BNetzA Ladesäulenregister](ev-sources.md#bnetza) | 24 h | nothing |
| Germany | `EV_DE_OCPDB` | [Open ChargePoint Database](ev-sources.md#ocpdb), with prices | 6 h | nothing |
| Germany, opt-in | `EV_DE_BNETZA` | [BNetzA Ladesäulenregister](ev-sources.md#bnetza) | 24 h | nothing |

Spain and Germany each use exactly one EV source at a time, never both:
Spain and Germany each use exactly one EV source at a time:

- **Spain** uses Mapa REVE when `PUMPERLY_REVE_API_KEY` is set, and Open Charge Map otherwise.
- **Germany** uses the BNetzA register by default. Set `PUMPERLY_DE_EV_SOURCE=ocm` to use Open Charge Map instead.
- **Germany** uses OCPDB by default. Set `PUMPERLY_DE_EV_SOURCE=bnetza` for the BNetzA register, or `ocm` for Open Charge Map.

Without `PUMPERLY_OCM_API_KEY`, the Open Charge Map scrapers still run on schedule but write nothing. German chargers from BNetzA still load, because that source needs no key.
Without `PUMPERLY_OCM_API_KEY`, the Open Charge Map scrapers still run on schedule but write nothing. German chargers from OCPDB or BNetzA still load, because those sources need no key.

EV chargers have no prices. None of the three sources publishes a per-litre price, and per-kWh tariffs do not fit the price model. [EV charging sources](ev-sources.md) covers each source in detail.
Only German chargers from OCPDB have prices: the cheapest ad-hoc tariff per kWh, with any session and per-minute fees. Chargers from every other source have none. [EV charging sources](ev-sources.md) covers each source in detail.

## Community datasets {#community-datasets}

Places with no source to scrape can be added as committed data files under [`src/scrapers/data/`](https://github.com/GeiserX/Pumperly/tree/main/src/scrapers/data). Each dataset registers its own scraper, keyed `STATIC_<SOURCE>`, and writes only its own rows. The repository ships with no datasets registered yet. [`src/scrapers/data/README.md`](https://github.com/GeiserX/Pumperly/blob/main/src/scrapers/data/README.md) explains the format, and [Adding a country](adding-a-country.md) covers the wider process.

## Choosing what runs {#choosing-what-runs}

By default every fuel scraper runs, including both Australian states. Each country also gets one EV source: Open Charge Map everywhere, including the United States, except that Germany uses the BNetzA register and Spain uses Mapa REVE when its key is set. These settings narrow that down:
By default every fuel scraper runs, including both Australian states. Each country also gets one EV source: Open Charge Map everywhere, including the United States, except that Germany uses OCPDB and Spain uses Mapa REVE when its key is set. These settings narrow that down:

| Setting | Effect |
|---|---|
| `PUMPERLY_ENABLED_COUNTRIES` | Comma-separated country codes. Each code enables its fuel scraper and its EV scraper. `US` enables EV chargers for the United States. |
| `PUMPERLY_EV_ENABLED=0` | Turns off every EV scraper, including Mapa REVE and BNetzA. |
| `PUMPERLY_EV_ENABLED=0` | Turns off every EV scraper, including Mapa REVE, OCPDB and BNetzA. |
| `PUMPERLY_SCRAPE_INTERVAL_<KEY>` | Sets one scraper's interval in hours, using the key from the tables above. `0` stops that scraper. |
| `PUMPERLY_SCRAPE_INTERVAL_HOURS` | Sets the interval for every scraper that has no per-key override. `0` stops all automatic scraping. |

Expand Down
Loading
Loading