Development and deployment

There are various scripts to help setup MapComplete for deployment and develop-deployment.

This documents attempts to shed some light on these scripts.

Note: these scripts change every now and then - if the documentation here is incorrect or you run into troubles, do leave a message in the issue tracker

Architecture overview

At its core, MapComplete is a static (!) website. There are no servers to host.

The data is fetched from Overpass/OSM/Wikidata/Wikipedia/Mapillary/... and written there directly. This means that any static file server will do to create a self-hosted version of MapComplete.

Development

Windows users: All scripts are made for linux devices. Use the Ubuntu terminal for Windows (or even better - make the switch ;) ). If you are using Visual Studio Code you can use a WSL Remote window, or use the Devcontainer (see more details later).

To develop and build MapComplete, you

Make a fork and clone the repository. (We recommend a shallow clone with git clone --filter=blob:none <repo>)
Install the nodejs version specified in .tool-versions
- On linux: install npm first sudo apt install npm, then install n using npm: npm install -g n, which can then install node with n install <node-version>
- You can use asdf to manage your runtime versions.
Install npm. Linux: sudo apt install npm (or your favourite package manager), Windows: install nodeJS: https://nodejs.org/en/download/
Run npm run init which …
- runs npm install
- generates some additional dependencies and files
Run npm run start to host a local testversion at http://localhost:1234/index.html
By default, a landing page with available themes is served. In order to load a single theme, use layout=themename or userlayout=true#<layout configuration> as Query parameter. Note that the shorter URLs ( e.g. bookcases.html, aed.html, ...) don't exist on the development version.

Development using Windows

For Windows you can use the devcontainer, or the WSL subsystem.

Raw installation

Clone the repo
Install npm and install ts-node globally with npm install -g ts-node
Execute npm run init. It will install and build some assets
Run npm run start to start the dev server

To use the WSL in Visual Studio Code:

Install a WSL Distribution (e.g. Ubuntu)
Install basic dependencies sudo apt install make g++
Install NVM wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
Use NVM to install node 16.x: nvm install 16.9.1
Activate the node version: nvm use 16.9.1
Install npm using sudo apt install npm.
Install the Remote - WSL extension and it's dependencies.
Open a remote WSL window using the button in the bottom left.
Make a fork and clone the repository.
Run npm run init and generate some additional dependencies and generated files. Note that it'll install the dependencies too
Run npm run start to host a local testversion at http://localhost:1234/index.html
By default, a landing page with available themes is served. In order to load a single theme, use layout=themename or userlayout=true#<layout configuration> as Query parameter. Note that the shorter URLs ( e.g. bookcases.html, aed.html, ...) don't exist on the development version.

To use WSL without Visual Studio Code you can replace steps 7 and 8 by opening up a WSL terminal

On mac

Install the `Command line tools for XCode which you can find here. You might need an apple dev account for this.

Automatic deployment

Currently, the master branch is automatically deployed to 'mapcomplete.osm.be' by a github action.

Every branch is automatically built (upon push) to 'pietervdvn.github.io/mc/' by a github action.

Deploying a fork

A script creates a webpage for every theme automatically, with some customizations in order to:

to have shorter urls
have individual social images
have individual web manifests

This script can be invoked with npm run prepare-deploy

If you want to deploy your fork:

npm run prepare-deploy
npm run build
Copy the entire dist folder to where you host your website. Visiting index.html gives you the landing page, visiting yourwebsite/<theme> should bring you to the appropriate theme.

Weird errors

Try removing node_modules, package-lock.json and .cache

Misc setup

~~The json-git-merger is used to quickly merge translation files, documentation here.~~ This merge driver is broken and would sometimes drop new questions or duplicate them... Not a good idea!

Overview of package.json-scripts

increase-memory: this is a big (and memory-intensive) project to build and run, so we give nodejs some more RAM.
start: start a development server.
test: run the unit tests
init: Generates and downloads various assets which are needed to compile
generate:editor-layer-index: downloads the editor-layer-index-json from osmlab.github.io
generate:images: compiles the SVG's into an asset
generate:translations: compiles the translation file into a javascript file
generate:layouts: uses index.html as template to create all the theme index pages. You'll want to run clean when done
generate:docs: generates various documents, such as information about available metatags, information to put on the OSM-wiki,...
generate:report: downloads statistics from OsmCha, compiles neat graphs
generate:cache:speelplekken: creates an offline copy of all the data required for one specific (paid for) theme
generate:layeroverview: reads all the theme- and layerconfigurations, compiles them into a single JSON.
reset:layeroverview: if something is wrong with the layeroverview, creates an empty one
generate:licenses: compiles all the license info of images into a single json
optimize:images: attempts to make smaller pngs - optional to run before a deployment
generate: run all the necesary generate-scripts
build: actually bundle all the files into a single dist/-folder
prepare-deploy: create the layouts
deploy:staging,deploy:pietervdvn, deploy:production: deploy the latest code on various locations
lint: get depressed by the amount of warnings
clean: remove some generated files which are annoying in the repo

6.9 KiB Raw Blame History