Merge branch 'master' into develop
This commit is contained in:
commit
8c1efc5888
4 changed files with 111 additions and 46 deletions
74
Docs/Development_deployment.md
Normal file
74
Docs/Development_deployment.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
|
||||
Development and deployment
|
||||
==========================
|
||||
|
||||
There are various scripts to help setup MapComplete for deployment and develop-deployment.
|
||||
|
||||
This documents attempts to shed some light on these scripts.
|
||||
|
||||
Note: these scripts change every now and then - if the documentation here is incorrect or you run into troubles, do leave a message in [the issue tracker](https://github.com/pietervdvn/MapComplete/issues)
|
||||
|
||||
Architecture overview
|
||||
---------------------
|
||||
|
||||
At its core, MapComplete is a static (!) website. There are no servers to host.
|
||||
|
||||
The data is fetched from Overpass/OSM/Wikidata/Wikipedia/Mapillary/... and written there directly. This means that any static file server will do to create a self-hosted version of MapComplete.
|
||||
|
||||
Development
|
||||
-----------
|
||||
|
||||
To develop and build MapComplete, yo
|
||||
|
||||
0. Make sure you have a recent version of nodejs - at least 12.0, preferably 15
|
||||
0. Make a fork and clone the repository.
|
||||
1. Install `npm`. Linux: `sudo apt install npm` (or your favourite package manager), Windows: install nodeJS: https://nodejs.org/en/download/
|
||||
2. Run `npm install` to install the package dependencies
|
||||
3. Run `npm run init` and generate some additional dependencies and generated files
|
||||
4. Run `npm run start` to host a local testversion at http://localhost:1234/index.html
|
||||
5. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename` or `userlayout=true#<layout configuration>` as [Query parameter](URL_Parameters.md). Note that the shorter URLs (e.g. `bookcases.html`, `aed.html`, ...) _don't_ exist on the development version.
|
||||
|
||||
|
||||
Deploying a fork
|
||||
----------------
|
||||
|
||||
A script creates a webpage for every theme automatically, with some customizations in order to:
|
||||
|
||||
- to have shorter urls
|
||||
- have individual social images
|
||||
- have individual web manifests
|
||||
|
||||
|
||||
This script can be invoked with `npm run prepare-deploy`
|
||||
|
||||
If you want to deploy your fork:
|
||||
|
||||
0. `npm run prepare-deploy`
|
||||
1. `npm run build`
|
||||
2. Copy the entire `dist` folder to where you host your website. Visiting `index.html` gives you the landing page, visiting `yourwebsite/<theme>` should bring you to the appropriate theme.
|
||||
|
||||
Overview of package.json-scripts
|
||||
--------------------------------
|
||||
|
||||
- `increase-memory`: this is a big (and memory-intensive) project to build and run, so we give nodejs some more RAM.
|
||||
- `start`: start a development server.
|
||||
- `test`: run the unit tests
|
||||
- `init`: Generates and downloads various assets which are needed to compile
|
||||
- `generate:editor-layer-index`: downloads the editor-layer-index-json from osmlab.github.io
|
||||
- `generate:images`: compiles the SVG's into an asset
|
||||
- `generate:translations`: compiles the translation file into a javascript file
|
||||
- `generate:layouts`: uses `index.html` as template to create all the theme index pages. You'll want to run `clean` when done
|
||||
- `generate:docs`: generates various documents, such as information about available metatags, information to put on the [OSM-wiki](https://wiki.openstreetmap.org/wiki/MapComplete),...
|
||||
- `generate:cache:speelplekken`: creates an offline copy of all the data required for one specific (paid for) theme
|
||||
- `generate:layeroverview`: reads all the theme- and layerconfigurations, compiles them into a single JSON.
|
||||
- `reset:layeroverview`: if something is wrong with the layeroverview, creates an empty one
|
||||
- `generate:licenses`: compiles all the license info of images into a single json
|
||||
- `optimize:images`: attempts to make smaller pngs - optional to run before a deployment
|
||||
- `generate`: run all the necesary generate-scripts
|
||||
- `build`: actually bundle all the files into a single `dist/`-folder
|
||||
- `prepare-deploy`: create the layouts
|
||||
- `deploy:staging`,`deploy:pietervdvn`, `deploy:production`: deploy the latest code on various locations
|
||||
- `lint`: get depressed by the amount of warnings
|
||||
- `clean`: remove some generated files which are annoying in the repo
|
||||
|
||||
|
||||
21
README.md
21
README.md
|
|
@ -60,7 +60,7 @@ A typical user journey would be:
|
|||
* Note that _all messages_ must be read before being able to add a point.
|
||||
* In other words, sending a message to a misbehaving MapComplete user acts as having a **zero-day-block**. This is added deliberately to make sure new users _have_ to read feedback from the community.
|
||||
|
||||
4. At 50 changesets, the [personal layout](https://pietervdvn.github.io/MapComplete/personal.html) is advertised. The personal theme is a theme where contributors can pick layers from all the offical themes. Note that the personal theme is always available.
|
||||
4. At 50 changesets, the [personal layout](https://pietervdvn.github.io/MapComplete/personal.html) is advertised. The personal theme is a theme where contributors can pick layers from all the official themes. Note that the personal theme is always available.
|
||||
|
||||
5. At 200 changesets, the tags become visible when answering questions and when adding a new point from a preset. This is to give more control to power users and to teach new users the tagging scheme
|
||||
|
||||
|
|
@ -77,21 +77,8 @@ I love it to see where the project ends up. You are free to reuse the software (
|
|||
|
||||
## Dev
|
||||
|
||||
To develop:
|
||||
To develop or deploy a version of MapComplete, have a look [to the guide](Docs/Development_deployment.md).
|
||||
|
||||
0. Make sure you have a recent version of nodejs - at least 12.0, preferably 15
|
||||
0. Make a fork and clone the repository.
|
||||
1. Install `npm`. Linux: `sudo apt install npm` (or your favourite package manager), Windows: install nodeJS: https://nodejs.org/en/download/
|
||||
2. Run `npm install` to install the package dependencies
|
||||
3. Run `npm run generate` and `npm run generate:editor-layer-index` to generate some additional dependencies
|
||||
4. Run `npm run start` to host a local testversion at http://localhost:1234/index.html
|
||||
5. By default, the 'bookcases'-theme is loaded. In order to load another theme, use `layout=themename` or `userlayout=true#<layout configuration>`. Note that the custom URLs (e.g. `bookcases.html`, `aed.html`, ...) _don't_ exist on the development version. (These are automatically generated from a template on the server).
|
||||
|
||||
To deploy:
|
||||
|
||||
0. `rm -rf dist/` to remove the local build
|
||||
1. `npm run build`
|
||||
2. Copy the entire `dist` folder to where you host your website. Visiting `index.html` gives you the website
|
||||
|
||||
## Translating MapComplete
|
||||
|
||||
|
|
@ -131,7 +118,7 @@ Images are fetched from:
|
|||
|
||||
Images are uploaded to Imgur, as their API was way easier to handle. The URL is written into the changes.
|
||||
|
||||
The idea is that once in a while, the images are transfered to wikipedia or that we hook up wikimedia directly (but I need some help in getting their API working).
|
||||
The idea is that once in a while, the images are transferred to wikipedia or that we hook up wikimedia directly (but I need some help in getting their API working).
|
||||
|
||||
### Uploading changes
|
||||
|
||||
|
|
@ -149,7 +136,7 @@ All documentation can be found in [here](Docs/)
|
|||
|
||||
Privacy is important, we try to leak as little information as possible.
|
||||
All major personal information is handled by OSM.
|
||||
Geolocation is available on mobile only throught hte device's GPS location (so no geolocation is sent of to Google).
|
||||
Geolocation is available on mobile only through the device's GPS location (so no geolocation is sent of to Google).
|
||||
|
||||
TODO: erase cookies of third party websites and API's
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@
|
|||
"increase-memory": "export NODE_OPTIONS=--max_old_space_size=4096",
|
||||
"start": "ts-node scripts/generateLayerOverview.ts --no-fail && npm run increase-memory && parcel *.html UI/** Logic/** assets/** assets/**/** assets/**/**/** vendor/* vendor/*/*",
|
||||
"test": "ts-node test/Tag.spec.ts && ts-node test/TagQuestion.spec.ts && ts-node test/ImageSearcher.spec.ts && ts-node test/ImageAttribution.spec.ts && ts-node test/Theme.spec.ts",
|
||||
"init": "npm run generate:editor-layer-index && npm run generate:licenses && npm run generate:layeroverview && npm run generate:layouts && npm run clean",
|
||||
"init": "npm run generate && npm run generate:layouts && npm run clean",
|
||||
"generate:editor-layer-index": "cd assets/ && wget https://osmlab.github.io/editor-layer-index/imagery.geojson --output-document=editor-layer-index.json",
|
||||
"generate:images": "ts-node scripts/generateIncludedImages.ts",
|
||||
"generate:translations": "ts-node scripts/generateTranslations.ts",
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
import {Utils} from "../Utils";
|
||||
import {lstatSync, readdirSync, readFileSync, writeFileSync, unlinkSync} from "fs";
|
||||
import {lstatSync, readdirSync, readFileSync, writeFileSync, unlinkSync, existsSync, mkdir, mkdirSync} from "fs";
|
||||
import SmallLicense from "../Models/smallLicense";
|
||||
import ScriptUtils from "./ScriptUtils";
|
||||
|
||||
|
|
@ -13,28 +13,29 @@ Utils.runningFromConsole = true;
|
|||
function generateLicenseInfos(paths: string[]): SmallLicense[] {
|
||||
const licenses = []
|
||||
for (const path of paths) {
|
||||
try{
|
||||
|
||||
|
||||
const parsed = JSON.parse(readFileSync(path, "UTF-8"))
|
||||
if (Array.isArray(parsed)) {
|
||||
const l: SmallLicense[] = parsed
|
||||
for (const smallLicens of l) {
|
||||
smallLicens.path = path.substring(0, path.length - "license_info.json".length) + smallLicens.path
|
||||
}
|
||||
licenses.push(...l)
|
||||
} else {
|
||||
const smallLicens: SmallLicense = parsed;
|
||||
/*if(parsed.license === "CC-BY"){
|
||||
console.log("Rewriting ", path)
|
||||
parsed.license === "CC-BY 4.0"
|
||||
writeFileSync(path, JSON.stringify(smallLicens, null, " "))
|
||||
}*/
|
||||
try {
|
||||
|
||||
smallLicens.path = path.substring(0, 1 + path.lastIndexOf("/")) + smallLicens.path
|
||||
licenses.push(smallLicens)
|
||||
}}catch(e){
|
||||
console.error("Error: ",e, "while handling",path)
|
||||
|
||||
const parsed = JSON.parse(readFileSync(path, "UTF-8"))
|
||||
if (Array.isArray(parsed)) {
|
||||
const l: SmallLicense[] = parsed
|
||||
for (const smallLicens of l) {
|
||||
smallLicens.path = path.substring(0, path.length - "license_info.json".length) + smallLicens.path
|
||||
}
|
||||
licenses.push(...l)
|
||||
} else {
|
||||
const smallLicens: SmallLicense = parsed;
|
||||
/*if(parsed.license === "CC-BY"){
|
||||
console.log("Rewriting ", path)
|
||||
parsed.license === "CC-BY 4.0"
|
||||
writeFileSync(path, JSON.stringify(smallLicens, null, " "))
|
||||
}*/
|
||||
|
||||
smallLicens.path = path.substring(0, 1 + path.lastIndexOf("/")) + smallLicens.path
|
||||
licenses.push(smallLicens)
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("Error: ", e, "while handling", path)
|
||||
}
|
||||
|
||||
}
|
||||
|
|
@ -195,6 +196,9 @@ const contents = ScriptUtils.readDirRecSync("./assets")
|
|||
const licensePaths = contents.filter(entry => entry.indexOf("license_info.json") >= 0)
|
||||
const licenseInfos = generateLicenseInfos(licensePaths);
|
||||
|
||||
if (!existsSync("./assets/generated")) {
|
||||
mkdirSync("./assets/generated")
|
||||
}
|
||||
|
||||
writeFileSync("./assets/generated/license_info.json", JSON.stringify(licenseInfos, null, " "))
|
||||
|
||||
|
|
@ -203,13 +207,13 @@ const missingLicenses = missingLicenseInfos(licenseInfos, artwork)
|
|||
const invalidLicenses = licenseInfos.filter(l => (l.license ?? "") === "").map(l => `License for artwork ${l.path} is empty string or undefined`)
|
||||
for (const licenseInfo of licenseInfos) {
|
||||
for (const source of licenseInfo.sources) {
|
||||
if(source == ""){
|
||||
invalidLicenses.push("Invalid license: empty string in "+JSON.stringify(licenseInfo))
|
||||
if (source == "") {
|
||||
invalidLicenses.push("Invalid license: empty string in " + JSON.stringify(licenseInfo))
|
||||
}
|
||||
try{
|
||||
try {
|
||||
new URL(source);
|
||||
}catch{
|
||||
invalidLicenses.push("Not a valid URL: "+source)
|
||||
} catch {
|
||||
invalidLicenses.push("Not a valid URL: " + source)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -218,7 +222,7 @@ if (process.argv.indexOf("--prompt") >= 0 || process.argv.indexOf("--query") >=
|
|||
}
|
||||
if (missingLicenses.length > 0) {
|
||||
const msg = `There are ${missingLicenses.length} licenses missing and ${invalidLicenses.length} invalid licenses.`
|
||||
console.log( missingLicenses.concat(invalidLicenses).join("\n"))
|
||||
console.log(missingLicenses.concat(invalidLicenses).join("\n"))
|
||||
console.error(msg)
|
||||
if (process.argv.indexOf("--report") >= 0) {
|
||||
console.log("Writing report!")
|
||||
|
|
|
|||
Loading…
Reference in a new issue