Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config.example.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
"MAP_INITIAL_LON": 2.5,
"PDM_TILES_URL": "http://localhost:7800",
"DB_USE_IMPOSM_UPDATE": true,
"USE_SOFT_DATES": false,
"WORK_DIR": "/tmp/pdm",
"GEOJSON_BOUNDS": {
"type": "Polygon",
Expand Down
2 changes: 1 addition & 1 deletion db/01_setup_schema.sql
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ BEGIN
LIMIT 1;

id := 'best_contributor';
name := 'N°1 des contributions';
name := 'Top contributor';
acquired := result_userid = the_userid;

IF acquired THEN
Expand Down
6 changes: 4 additions & 2 deletions db/20_changes_update.js
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,9 @@ Object.values(projects).forEach(project => {
if (projectLength > 0){
projectsQry = `${projectsQry.substring(0, projectsQry.length-1)} ON CONFLICT (project_id) DO UPDATE SET start_date=EXCLUDED.start_date, end_date=EXCLUDED.end_date`;
pgPool.query(projectsQry, (err, res) => {
if (err){
if(err?.message?.includes("cannot affect row a second time")) {
throw new Error(`Error when installing projects: ${err}\n\nMake sure all projects have a distinct id, query was: ${projectsQry}`);
} else if (err) {
throw new Error(`Error when installing projects: ${err}`);
}
console.log(projectLength+" project(s) installed");
Expand Down Expand Up @@ -494,7 +496,7 @@ Object.values(projects).forEach(project => {
script += `
echo " => [\$((\$(date -d now +%s) - \$process_start_t0))s] Seek for all changes related to selected features and convert to OPL"
rm -f "${oplProject}"
osmium getid ${getIdOptions} "\$history_osh" -I "${oshProjectTags}" -f opl,history=true -o "${oplProject}"
osmium getid ${getIdOptions} "\$history_osh" -I "${oshProjectTags}" -f opl,history=true -o "${oplProject}" || { echo "osmium getid failed, check ${oshProjectTags}"; exit 1; }
rm -f "${csvFeatures}" "${csvMembers}" "${oshProjectTags}"

${macroChangesCsv ("init", project, oplProject, csvFeatures, csvUsers, csvMembers, "\$process_start_ts", "\$process_end_tss")}
Expand Down
4 changes: 3 additions & 1 deletion docs/DEVELOP.fr.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ La configuration générale de l'outil est à renseigner dans `config.json`. Un
- `OSM_PBF_URL`: URL du fichier OSM.PBF (etat courant de la base, exemple `https://download.geofabrik.de/europe/france-latest.osm.pbf`). Ce fichier n'est pas concerné par le processus d'autorisation.
- `POLY_URL`: URL d'un fichier de polygone dans lequel les projets existent (exemple `https://download.geofabrik.de/europe/france.poly`) Ce fichier n'est pas concerné par le processus d'autorisation.
- `DB_USE_IMPOSM_UPDATE` : Active ou désactive l'intégration d'imposm3 (permet d'utiliser une base existante et tenue à jour par d'autres moyens, par défaut `true`)
- `USE_SOFT_DATES` : utiliser soft_start_date et soft_end_date (au lieu de start_date et end_date) pour déterminer si un projet est passé, en cours ou à venir
- `WORK_DIR` : dossier de téléchargement et stockage temporaire (doit pouvoir contenir le fichier OSH PBF, exemple `/tmp/pdm`)
- `OSM_URL` : instance OpenStreetMap à utiliser (exemple `https://www.openstreetmap.org`)
- `OSM_API_URL` : instance API OpenStreetMap à utiliser (exemple `https://api.openstreetmap.org`)
Expand Down Expand Up @@ -101,8 +102,9 @@ Les propriétés dans `info.json` sont les suivantes :
- `name` : identifiant de la mission (caractères autorisés : A-Z, 0-9, \_ et -)
- `title` : nom de la mission (assez court)
- `start_date` : date de début de la mission (format AAAA-MM-JJ)
- `end_date` : date de fin de la mission (format AAAA-MM-JJ)
- `soft_start_date`: date de début de la période de _forte_ animation communautaire (format AAAA-MM-JJ). Donnée purement à titre informatif, n'affecte pas le traitement des données.
- `soft_end_date`: date de fin de la période de _forte_ animation communautaire (format AAAA-MM-JJ). Donnée purement à titre informatif, n'affecte pas le traitement des données.
- `end_date` : date de fin de la mission (format AAAA-MM-JJ)
- `summary` : résumé de la mission
- `links` : définition des URL pour les liens vers des pages tierces (wiki OSM, forum OSM ou page de blog) avec ce format "osmwiki|osmblog|osmforum": "projetdumois.fr"
- `database.osmium_tag_filter` : filtre Osmium sur les tags à appliquer pour ne conserver que les objets OSM pertinents (par exemple `nwr/*:covid19`, [syntaxe décrite ici](https://osmcode.org/osmium-tool/manual.html#filtering-by-tags)). Il est possible d'enchaîner plusieurs filtres par & et en répétant l'indication de primitive à chaque niveau. L'opérateur != n'est pour l'instant pas pris en compte.
Expand Down
4 changes: 3 additions & 1 deletion docs/DEVELOP.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ The general configuration of the tool is to be filled in `config.json`. There is
- `OSM_PBF_URL`: URL of the OSM.PBF file (current state, example `https://download.geofabrik.de/europe/france-latest.osm.pbf`). This file isn't covered by authorization process.
- `POLY_URL`: URL of a polygon file holding the perimeter in which projects are considered (example `https://download.geofabrik.de/europe/france.poly`). This file isn't covered by authorization process.
- `DB_USE_IMPOSM_UPDATE` : enable or disabled Imposm3 integration (to use an existing database which would be maintained by other means, by default `true`)
- `USE_SOFT_DATES`: whether to use soft_start_date and soft_end_date (instead of start_date and end_date) to decide whether a projects is past, current or next
- `WORK_DIR`: download and temporary storage folder (must have capacity to store the OSH PBF file, example `/tmp/pdm`)
- `OSM_URL`: OpenStreetMap instance to use (example `https://www.openstreetmap.org`)
- `OSM_API_URL` : API OpenStreetMap instance to use (example `https://www.api.openstreetmap.org`)
Expand Down Expand Up @@ -101,8 +102,9 @@ The properties in `info.json` are as follows:
- `name`: mission identifier (authorized characters: A-Z, 0-9, \_ and -)
- `title`: name of the mission (short enough)
- `start_date`: start date of the mission (format YYYYY-MM-DD)
- `end_date`: end date of the mission (format YYYYY-MM-DD)
- `soft_start_date`: start date of the _strong_ community animation period (format YYYYY-MM-DD). This is only informational, it doesn't affect backend processing.
- `soft_end_date`: end date of the _strong_ community animation period (format YYYYY-MM-DD). This is only informational, it doesn't affect backend processing.
- `end_date`: end date of the mission (format YYYYY-MM-DD)
- `summary`: summary of the mission
- `links`: definition of the URLs for links to third party pages (OSM wiki, OSM forum or blog page) with this format "osmwiki|osmblog|osmforum": "projetdumois.fr"
- `database.osmium_tag_filter` : Osmium filter on the tags to be applied to keep only the relevant OSM objects (for example `nwr/*:covid19`, [syntax described here](https://osmcode.org/osmium-tool/manual.html#filtering-by-tags)). It is possible to list many filters using `&` character and same syntax.
Expand Down
16 changes: 16 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"bootstrap.native": "^4.2.0",
"chart.js": "^4.5.1",
"chartjs-adapter-moment": "^1.0.1",
"chartjs-plugin-annotation": "^3.1.0",
"compression": "^1.8.1",
"cors": "^2.8.5",
"express": "^5.2.0",
Expand Down
82 changes: 38 additions & 44 deletions website/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -332,24 +332,27 @@ app.get("/projects/:name", (req, res) => {
.concat(all.current.filter((p) => p.name !== req.params.name));
const isActive =
all.current.length > 0 &&
all.current.find((p) => p.name === req.params.name) !== undefined;
all.current.some((p) => p.name === req.params.name);
const isNext =
all.next && all.next.find((p) => p.name === req.params.name) !== undefined;
const isHardEnded = all.past.find(p => (
all.next && all.next.some((p) => p.name === req.params.name);
const isHardEnded = all.past.some(p => (
p.id === req.params.id
&& p.end_date != null
&& new Date(p.end_date + "T23:59:59Z").getTime() < Date.now()
)) !== undefined;
));

const RECENT_PAST_DAYS = 30,
recentPastThreshold = Date.now() - RECENT_PAST_DAYS * 24 * 60 * 60 * 1000;
const isRecentPast =
all.past &&
all.past.length > 0 &&
all.past.find(
(p) =>
p.name === req.params.name && p.end_date != null &&
new Date(p.end_date + "T23:59:59Z").getTime() >=
Date.now() - 30 * 24 * 60 * 60 * 1000,
) !== undefined;
all.past.some((p) =>{
const endDate = CONFIG.USE_SOFT_DATES && p.soft_end_date || p.end_date;
return p.name === req.params.name &&
endDate != null &&
new Date(endDate + "T23:59:59Z").getTime() >= recentPastThreshold;
});

res.render(
"pages/project",
Object.assign(
Expand Down Expand Up @@ -433,9 +436,10 @@ app.get("/projects/:name/stats", (req, res) => {
const osmUserAuthentified =
typeof req.query.osm_user === "string" &&
req.query.osm_user.trim().length > 0;
const startDate = CONFIG.USE_SOFT_DATES && p.soft_start_date || p.start_date;
const daysToKeep = (day) => {
if (
Date.now() - new Date(p.start_date).getTime() <
Date.now() - new Date(startDate).getTime() <
1000 * 60 * 60 * 24 * 60
) {
return true;
Expand All @@ -455,7 +459,7 @@ app.get("/projects/:name/stats", (req, res) => {
const params = {
item: ds.item,
class: ds.class,
start_date: p.start_date,
start_date: startDate,
country: ds.country,
};
return fetch(
Expand Down Expand Up @@ -565,39 +569,26 @@ app.get("/projects/:name/stats", (req, res) => {
`,
[p.id]
)
.then((results) => ({
chart: [
{
label: res.__("Count in OSM"),
data: results.rows.map((r) => ({ x: r.ts, y: r.amount })),
fill: false,
borderColor: "#388E3C",
lineTension: 0,
.then((results) => {
const firstItem = results.rows[0],
lastItem = results.rows.length > 0 ? results.rows[results.rows.length - 1] : null,
secondLastItem = results.rows.length > 1 ? results.rows[results.rows.length - 2] : null;
return {
osm_counts: results.rows,
"daily":{
"ts": lastItem?.ts,
"ts_start": firstItem?.ts,
"count": lastItem?.amount,
"added": firstItem && lastItem && lastItem.amount - firstItem.amount
},
],
"daily":{
"ts": results.rows.length > 0 &&
results.rows[results.rows.length - 1].ts,
"ts_start": results.rows[0].ts,
"count":results.rows.length > 0 &&
results.rows[results.rows.length - 1].amount,
"added":
results.rows.length > 0 &&
results.rows[results.rows.length - 1].amount -
results.rows[0].amount
},
"past": {
"ts": results.rows.length > 1 &&
results.rows[results.rows.length - 1].ts,
"ts_start": results.rows[0].ts,
"count": results.rows.length > 1 &&
results.rows[results.rows.length - 2].amount,
"added":
results.rows.length > 1 &&
results.rows[results.rows.length - 2].amount -
results.rows[0].amount
}
})),
"past": {
"ts": secondLastItem?.ts,
"ts_start": firstItem?.ts,
"count": secondLastItem?.amount,
"added": firstItem && secondLastItem && secondLastItem.amount - firstItem.amount
}
};
}),
);

// Current time point is only available if a live table is maintained
Expand Down Expand Up @@ -1121,6 +1112,9 @@ const authorized = {
"chartjs-adapter-moment": {
"chartjs-adapter-moment.js": "dist/chartjs-adapter-moment.min.js",
},
"chartjs-plugin-annotation": {
"annotation.js": "dist/chartjs-plugin-annotation.min.js",
},
"maplibre-gl": {
"maplibre-gl.js": "dist/maplibre-gl.js",
"maplibre-gl.css": "dist/maplibre-gl.css",
Expand Down
5 changes: 4 additions & 1 deletion website/locales/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -141,13 +141,16 @@
"Count in OSM": "Count in OSM",
"Number of objects per key": "Number of objects per key",
"%s added since %s": "%s added since %s",
"%s added (%s)": "%s added (%s)",
"%s currently in OSM": "%s currently in OSM",
"%s in OSM on the last update": "%s in OSM on the last update",
"Thank you!": "Thank you!",
"Thank you all!": "Thank you all!",
"Thanks to all the contributors for making this project a success. Without all of you, OpenStreetMap would be nothing!": "Thanks to all the contributors for making this project a success. Without all of you, OpenStreetMap would be nothing!",
"General": "General",
"Participated": "Participated",
"They have participated to the project": "They have participated to the project",
"Always present": "Always present",
"They have participated to all the projects": "They have participated to all the projects"
"They have participated to all the projects": "They have participated to all the projects",
"Top contributor": "Top contributor"
}
5 changes: 4 additions & 1 deletion website/locales/fr.json
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,9 @@
"Count in OSM": "Nombre dans OSM",
"Number of objects per key": "Nombre d'objets pour la clé",
"%s added since %s": "%s ajoutés depuis %s",
"%s added (%s)": "%s ajoutés (%s)",
"%s currently in OSM": "%s actuellement dans OSM",
"%s in OSM on the last update": "%s dans OSM lors de la dernière mise à jour",
"Thank you!": "Merci!",
"Thank you all!": "Merci à toutes et tous!",
"Thanks to all the contributors for making this project a success. Without all of you, OpenStreetMap would be nothing!": "Merci à l'ensemble des personnes contributrices pour la réussite de ce projet. Sans vous, OpenStreetMap ne serait rien!",
Expand All @@ -153,5 +155,6 @@
"Near the podium": "Près du podium",
"1st place": "1ère place",
"2nd place": "2ème place",
"3rd place": "3ème place"
"3rd place": "3ème place",
"Top contributor": "N°1 des contributions"
}
11 changes: 7 additions & 4 deletions website/locales/it.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"Total": "Totale",
"Country": "Nazione",
"Region": "Regione",
"Department": "Provincia",
"Depart.": "Provincia",
"City": "Città",
"Find all data directly on OpenStreetMap website.": "Trova tutti i dati direttamente sul sito di OpenStreetMap.",
"Other projects": "Altri progetti",
Expand Down Expand Up @@ -140,8 +140,10 @@
"Discuss": "Discussione",
"Count in OSM": "Conteggio in OSM",
"Number of objects per key": "Numero di oggetti per chiave",
"%s added since %s": "%s aggiunti da allora %s",
"%s currently in OSM": "%s attualmente in OSM",
"%s added since %s": "%s aggiunti dal %s",
"%s added (%s)": "%s aggiunti (%s)",
"%s currently in OSM": "%s attualmente su OSM",
"%s in OSM on the last update": "%s su OSM all'ultimo aggiornamento",
"Thank you!": "Grazie!",
"Thank you all!": "Grazie a tutti!",
"Thanks to all the contributors for making this project a success. Without all of you, OpenStreetMap would be nothing!": "Grazie a tutti i contributori per aver reso questo progetto un successo! Senza di voi OpenStreetMap non sarebbe quello che è!",
Expand All @@ -161,5 +163,6 @@
"42+ points": "42+ punti",
"70+ points": "70+ punti",
"100+ points": "100+ punti",
"500+ points": "500+ punti"
"500+ points": "500+ punti",
"Top contributor": "N°1 dei contributi"
}
1 change: 1 addition & 0 deletions website/templates/common/head.pug
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ script(src="/lib/bootstrap.native/bootstrap.js")
script(src="/lib/moment/moment.js")
script(src="/lib/chart.js/chart.js")
script(src="/lib/chartjs-adapter-moment/chartjs-adapter-moment.js")
script(src="/lib/chartjs-plugin-annotation/annotation.js")
script(src="/lib/osm-auth/osmauth.js")
script(src="/lib/osm-request/osmrequest.js")

Expand Down
Loading