2021-09-09 00:05:51 +02:00
Development and deployment
==========================
There are various scripts to help setup MapComplete for deployment and develop-deployment.
This documents attempts to shed some light on these scripts.
Note: these scripts change every now and then - if the documentation here is incorrect or you run into troubles, do
leave a message in [the issue tracker ](https://github.com/pietervdvn/MapComplete/issues )
Architecture overview
---------------------
At its core, MapComplete is a static (!) website. There are no servers to host.
The data is fetched from Overpass/OSM/Wikidata/Wikipedia/Mapillary/... and written there directly. This means that any
static file server will do to create a self-hosted version of MapComplete.
Development
-----------
**Windows users**: All scripts are made for linux devices. Use the Ubuntu terminal for Windows (or even better - make
the switch ;) ). If you are using Visual Studio Code you can use
a [WSL Remote ](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl ) window, or use the
Devcontainer (see more details later).
2023-03-09 17:11:44 +01:00
You need at least 3Gb RAM available to run MapComplete, but you'll preferably have 8GB of free RAM available.
2023-01-03 12:54:23 +01:00
2021-09-09 00:05:51 +02:00
To develop and build MapComplete, you
2021-05-07 02:18:52 +02:00
2022-05-01 18:46:05 +02:00
0. Make a fork and clone the repository. (We recommend a shallow clone with `git clone --filter=blob:none <repo>` )
2023-03-09 17:11:44 +01:00
1. Install `python3` if you do not have it already - On linux: `sudo apt install python3`
2. Install `nvm` to easily install node:
- `wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash`
- Restart your terminal
- Run `nvm install` and `nvm use` to install and use the correct version of node. (_Note: nvm might complain that the relevant version is not yet installed. It'll have it installed only for the current user account but not system-wide - which is fine)
4. Run `npm run init` (including **run** , not ~~`npm init`~~ )which …
- runs `npm ci` for you
- generates some additional dependencies and files
- does various housekeeping and setup. This can take a few minutes the first time as some pngs need to be created
5. Run `npm run start` to host a local testversion at http://localhost:1234/
6. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
2021-09-09 00:05:51 +02:00
or `userlayout=true#<layout configuration>` as [Query parameter ](URL_Parameters.md ). Note that the shorter URLs (
e.g. `bookcases.html` , `aed.html` , ...) _don't_ exist on the development version.
2021-07-13 13:57:31 +02:00
2023-03-09 17:11:44 +01:00
The previous instructions were tested on 2023-03-09 on a Ubuntu 22.04 machine
2021-09-09 00:05:51 +02:00
Development using Windows
------------------------
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
For Windows you can use the devcontainer, or the WSL subsystem.
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
To use the devcontainer in Visual Studio Code:
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
0. Make sure you have installed
the [Remote - Containers ](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers )
extension and it's dependencies.
2021-07-13 13:57:31 +02:00
1. Make a fork and clone the repository.
2. After cloning, Visual Studio Code will ask you if you want to use the devcontainer.
3. Then you can either clone it again in a volume (for better performance), or open the current folder in a container.
4. By now, you should be able to run `npm run start` to host a local testversion at http://localhost:1234/index.html
2021-09-09 00:05:51 +02:00
5. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
or `userlayout=true#<layout configuration>` as [Query parameter ](URL_Parameters.md ). Note that the shorter URLs (
e.g. `bookcases.html` , `aed.html` , ...) _don't_ exist on the development version.
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
To use the WSL in Visual Studio Code:
2021-07-13 13:57:31 +02:00
0. Make sure you have installed the [Remote - WSL]() extension and it's dependencies.
1. Open a remote WSL window using the button in the bottom left.
2. Make a fork and clone the repository.
3. Install `npm` using `sudo apt install npm` .
2021-09-09 00:05:51 +02:00
4. Run `npm run init` and generate some additional dependencies and generated files. Note that it'll install the
dependencies too
2021-07-13 13:57:31 +02:00
5. Run `npm run start` to host a local testversion at http://localhost:1234/index.html
2021-09-09 00:05:51 +02:00
6. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
or `userlayout=true#<layout configuration>` as [Query parameter ](URL_Parameters.md ). Note that the shorter URLs (
e.g. `bookcases.html` , `aed.html` , ...) _don't_ exist on the development version.
2022-07-30 15:07:55 +02:00
Dependencies
------------
2022-07-13 17:58:23 +02:00
`make` , `python3` `g++`
2022-07-30 15:07:55 +02:00
(Nix users may run `nix-env -iA nixos.gnumake nixos.gdc nixos.python3` )
2022-07-13 17:58:23 +02:00
2021-09-09 00:05:51 +02:00
Automatic deployment
--------------------
Currently, the master branch is automatically deployed to 'mapcomplete.osm.be' by a github action.
2021-05-07 02:18:52 +02:00
2021-09-09 00:05:51 +02:00
Every branch is automatically built (upon push) to 'pietervdvn.github.io/mc/< branchname > ' by a github action.
2021-05-07 02:18:52 +02:00
2021-06-04 17:00:36 +02:00
2021-09-09 00:05:51 +02:00
Deploying a fork
----------------
2021-06-04 17:00:36 +02:00
2021-09-09 00:05:51 +02:00
A script creates a webpage for every theme automatically, with some customizations in order to:
- to have shorter urls
- have individual social images
- have individual web manifests
This script can be invoked with `npm run prepare-deploy`
2021-05-07 02:18:52 +02:00
If you want to deploy your fork:
0. `npm run prepare-deploy`
1. `npm run build`
2021-09-09 00:05:51 +02:00
2. Copy the entire `dist` folder to where you host your website. Visiting `index.html` gives you the landing page,
visiting `yourwebsite/<theme>` should bring you to the appropriate theme.
2021-05-07 13:59:40 +02:00
Weird errors
------------
Try removing `node_modules` , `package-lock.json` and `.cache`
2021-10-01 04:59:22 +02:00
Misc setup
----------
2022-01-26 21:40:38 +01:00
~~The json-git-merger is used to quickly merge translation
files, [documentation here ](https://github.com/jonatanpedersen/git-json-merge#single-project--directory ).~~
2021-11-08 03:22:00 +01:00
This merge driver is broken and would sometimes drop new questions or duplicate them... Not a good idea!
2021-10-01 04:59:22 +02:00
2021-09-09 00:05:51 +02:00
Overview of package.json-scripts
--------------------------------
- `increase-memory` : this is a big (and memory-intensive) project to build and run, so we give nodejs some more RAM.
- `start` : start a development server.
- `test` : run the unit tests
- `init` : Generates and downloads various assets which are needed to compile
- `generate:editor-layer-index` : downloads the editor-layer-index-json from osmlab.github.io
- `generate:images` : compiles the SVG's into an asset
- `generate:translations` : compiles the translation file into a javascript file
- `generate:layouts` : uses `index.html` as template to create all the theme index pages. You'll want to run `clean` when
done
- `generate:docs` : generates various documents, such as information about available metatags, information to put on
the [OSM-wiki ](https://wiki.openstreetmap.org/wiki/MapComplete ),...
- `generate:report` : downloads statistics from OsmCha, compiles neat graphs
- `generate:cache:speelplekken` : creates an offline copy of all the data required for one specific (paid for) theme
- `generate:layeroverview` : reads all the theme- and layerconfigurations, compiles them into a single JSON.
- `reset:layeroverview` : if something is wrong with the layeroverview, creates an empty one
- `generate:licenses` : compiles all the license info of images into a single json
- `optimize:images` : attempts to make smaller pngs - optional to run before a deployment
- `generate` : run all the necesary generate-scripts
- `build` : actually bundle all the files into a single `dist/` -folder
- `prepare-deploy` : create the layouts
- `deploy:staging` ,`deploy:pietervdvn`, `deploy:production` : deploy the latest code on various locations
- `lint` : get depressed by the amount of warnings
- `clean` : remove some generated files which are annoying in the repo
