A Map Compiler for OSM data

(book chp 7.3)

This page will demonstrate the usage of the OSM compiler
with an example OSM file.

p 81: A basic structure of an OpenStreetMap compiler is provided in the resources folder OSM.compiler ... [download link] ...

 

You should move this folder to a large hard disk, where you can place your downloaded (input) and processed map data (output). The batch files were created for Windows systems and can be replaced with Unix shells. The main goal is to make each compiler step accessible via a command line. This way, the steps can be invoked from a higher-level script to wrap the entire production process in one place—later. The tools folder holds third-party tools; you should download the latest versions for best results.

1. Osmosis: unzip bz2 to osm

unzip hamburg_germany.osm.bz2 to hamburg_germany.osm
Mrz 29, 2015 3:06:02 PM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Osmosis Version 0.43.1
Mrz 29, 2015 3:06:03 PM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Preparing pipeline.
Mrz 29, 2015 3:06:03 PM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Launching pipeline execution.
Mrz 29, 2015 3:06:03 PM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Pipeline executing, waiting for completion.
Mrz 29, 2015 3:09:27 PM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Pipeline complete.
Mrz 29, 2015 3:09:27 PM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Total execution time: 204564 milliseconds.

85 MB - hamburg_germany.osm.bz2
1 GB - hamburg_germany.osm

2. JOSM: clip one square kilometer from downtown Hamburg

4,6 MB - hamburg_clip.osm

3. 

cleanup hamburg_germany.osm to hamburg_germany.clean.osm
read: F:\virtex\projects\OSM.compiler\deliveries\hamburg_germany.osm
write: F:\virtex\projects\OSM.compiler\deliveries\hamburg_germany.clean.osm
starting '-clean' at Sun Mar 29 19:01:19 CEST 2015
start parsing F:\virtex\projects\OSM.compiler\deliveries\hamburg_germany.osm

statistics 1GB hamburg_germany.osm <> 847 MB hamburg_germany.clean.osm
lines: 16748843 <> 16584222
nodes: 4795764 <> 4795764
ways: 867464 <> 867464
relations: 14523 <> 14523
ending '-clean' at Sun Mar 29 19:03:10 CEST 2015

simple and most effective

OSM downloads have become too big to be handled with interactive tools and developers have to apply command line tools to process data. The OSM parser introduced in the book basically a regular expression parser. It is aware of the basic OSM elements and designed not to break the XML integrity. This page will demonstrate the usage of the OSMparser to 'clean up' downloaded OSM files and shrink the contents down to a dedicated map application. The complete process can be modeled with a regular batch or shell script to make it reproducible on new data and for regular map data updates and releases. Finally many batch processes can be alligned in a larger map compilation process.
 
 

Download OSM file

The OSMparser can read unvompressed *.osm files containing specified XML data. For this tutorial osm data for the city of Hamburg in Germany will be used. As a starting point we download the clipped data from 
 
metro extracts "City-sized portions of OpenStreetMap, served weekly".
 
The unpacked file hamburg_germany.osm is bigger than a GByte and can not be loaded in JOSM and most Texteditors. 

Create OSM sample with JOSM

In order to peek into the file with interactive tools we will use JOSM to download about a square kilometer of Hamburgs City Center. Our clip hamburg_clip.osm has about 4-5 MB and can be viewed in a Texteditor during the process.

OSMparser.java or .class

supply java and precompiled class files .... (see sources)

Create Project Directory

To make the process transparent we'll place all files in a single directory. Something like OSM.compiler\products\HH. Move both osm files, the target and the sample file and the OSMparser class.
 
 
 
 
 
 
 
 
 
 
 « introduction xxx »