I installed Nominatim using full planet data from 2019-03-25 a couple of months ago, but initiated updates (as described at http://nominatim.org/release-docs/latest/admin/Import-and-Update/#updates) this week. Several diffs were applied at about 15-30 minutes per diff and then osm2pgsql got stuck on a diff only 68MB large.

I interrupted osm2pgsql after about 6 hours and restarted it and it has been running at 100% CPU for almost 20 hours already.

During the first (interrupted) run its last output was

 Processing: Node(238k 1.9k/s) Way(31k 0.16k/s) Relation(700 16.67/s)

while in the current run the count of processed relations is lower (570 vs 700)

 Processing: Node(238k 1.9k/s) Way(31k 0.16k/s) Relation(570 11.18/s)

Note, however, that the first run was performed without any output redirection, while the output of the current run is redirected to a file which probably means that the most recent progress message is simply not flushed yet.

Having copied the update command as is, I didn't provide the --osm2pgsql-cache option, so it is using the default value of 2000MB. But the fact that osm2pgsql is running at 100% CPU doesn't suggest that the low cache setting can be the root cause of this problem.

Examining the stack trace at different times suggests that osm2pgsql is processing a complex relation and is busy with a deep recursion inside osmium::area::detail::BasicAssembler::find_candidates():

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
std::_Construct<std::pair<osmium::area::detail::location_to_ring_map, bool>, std::pair<osmium::area::detail::location_to_ring_map, bool> > (__p=<optimized out>) at /usr/include/c++/7/bits/stl_construct.h:75
75      { ::new(static_cast<void*>(__p)) _T1(std::forward<_Args>(__args)...); }
#0  std::_Construct<std::pair<osmium::area::detail::location_to_ring_map, bool>, std::pair<osmium::area::detail::location_to_ring_map, bool> > (__p=<optimized out>) at /usr/include/c++/7/bits/stl_construct.h:75
#1  std::__uninitialized_copy<false>::__uninit_copy<std::move_iterator<std::pair<osmium::area::detail::location_to_ring_map, bool>*>, std::pair<osmium::area::detail::location_to_ring_map, bool>*> (__result=<optimized out>, __last=..., 
    __first=...) at /usr/include/c++/7/bits/stl_uninitialized.h:83
#2  std::uninitialized_copy<std::move_iterator<std::pair<osmium::area::detail::location_to_ring_map, bool>*>, std::pair<osmium::area::detail::location_to_ring_map, bool>*> (__result=<optimized out>, __last=..., __first=...)
    at /usr/include/c++/7/bits/stl_uninitialized.h:134
#3  std::__uninitialized_copy_a<std::move_iterator<std::pair<osmium::area::detail::location_to_ring_map, bool>*>, std::pair<osmium::area::detail::location_to_ring_map, bool>*, std::pair<osmium::area::detail::location_to_ring_map, bool> > (
    __result=<optimized out>, __last=..., __first=...) at /usr/include/c++/7/bits/stl_uninitialized.h:289
#4  std::__uninitialized_move_if_noexcept_a<std::pair<osmium::area::detail::location_to_ring_map, bool>*, std::pair<osmium::area::detail::location_to_ring_map, bool>*, std::allocator<std::pair<osmium::area::detail::location_to_ring_map, bool> > > (__alloc=..., __result=<optimized out>, __last=0x55b024d324e0, __first=0x55b024d31d20)
    at /usr/include/c++/7/bits/stl_uninitialized.h:312
#5  std::vector<std::pair<osmium::area::detail::location_to_ring_map, bool>, std::allocator<std::pair<osmium::area::detail::location_to_ring_map, bool> > >::_M_realloc_insert<osmium::area::detail::location_to_ring_map const&, bool> (
    this=this@entry=0x7ffd2f638998, __position=
  {first = {location = {m_x = 2000, m_y = 0, static undefined_coordinate = 2147483647}, ring_it = <error reading variable: Cannot access memory at address 0xf91>, start = 164}, second = 208}, __args#0=..., __args#1=@0x7ffd2f638980: true)
    at /usr/include/c++/7/bits/vector.tcc:424
#6  0x000055b01f0d28fe in std::vector<std::pair<osmium::area::detail::location_to_ring_map, bool>, std::allocator<std::pair<osmium::area::detail::location_to_ring_map, bool> > >::emplace_back<osmium::area::detail::location_to_ring_map const&, bool> (this=this@entry=0x7ffd2f638998, __args#0=..., __args#1=@0x7ffd2f638980: true)
    at /usr/include/c++/7/bits/vector.tcc:105
#7  0x000055b01f0d5d66 in osmium::area::detail::BasicAssembler::find_candidates (this=this@entry=0x7ffd2f63c280, 
    candidates=std::vector of length 0, capacity 0, loc_done=std::unordered_set with 62 elements = {...}, 
    xrings=std::vector of length 672, capacity 672 = {...}, cand=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/area/detail/basic_assembler.hpp:736

... more osmium::area::detail::BasicAssembler::find_candidates() frames ...
....
... more osmium::area::detail::BasicAssembler::find_candidates() frames ...

#68 0x000055b01f0d5e04 in osmium::area::detail::BasicAssembler::find_candidates (this=this@entry=0x7ffd2f63c280, 
    candidates=std::vector of length 0, capacity 0, loc_done=std::unordered_set with 62 elements = {...}, 
    xrings=std::vector of length 672, capacity 672 = {...}, cand=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/area/detail/basic_assembler.hpp:750
#69 0x000055b01f0dae1d in osmium::area::detail::BasicAssembler::join_connected_rings (this=this@entry=0x7ffd2f63c280, 
    open_ring_its=std::__cxx11::list = {...})
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/area/detail/basic_assembler.hpp:802
#70 0x000055b01f0dbcb4 in osmium::area::detail::BasicAssembler::create_rings_complex_case (
    this=this@entry=0x7ffd2f63c280)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/area/detail/basic_assembler.hpp:920
#71 0x000055b01f0dc842 in osmium::area::detail::BasicAssembler::create_rings (this=0x7ffd2f63c280)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/area/detail/basic_assembler.hpp:1087
#72 0x000055b01f0ce9b0 in osmium::area::GeomAssembler::operator() (out_buffer=..., ways_buffer=..., relation=..., 
    this=0x7ffd2f63c280) at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/area/geom_assembler.hpp:112
#73 geom::osmium_builder_t::get_wkb_multipolygon[abi:cxx11](osmium::Relation const&, osmium::memory::Buffer const&) (
    this=0x55b02047cf98, rel=..., ways=...) at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/osmium-builder.cpp:159
#74 0x000055b01f0a9c3d in output_gazetteer_t::process_relation (this=0x55b02047ca30, rel=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/output-gazetteer.cpp:704
#75 0x000055b01f05d080 in osmdata_t::relation_modify (this=<optimized out>, rel=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/osmdata.cpp:129
#76 0x000055b01f06ee82 in parse_osmium_t::relation (this=this@entry=0x7ffd2f63cab0, rel=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/parse-osmium.cpp:183
#77 0x000055b01f06fe70 in osmium::detail::apply_item_impl<parse_osmium_t&, osmium::memory::Item> (handler=..., item=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:68
#78 osmium::apply_item<osmium::memory::Item, parse_osmium_t&> (item=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:206
#79 osmium::apply<osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>, parse_osmium_t&> (end=..., it=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:220
#80 osmium::apply<osmium::io::Reader, parse_osmium_t&> (c=...)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:229
#81 parse_osmium_t::stream_file (this=0x7ffd2f63cab0, filename=..., fmt="auto")
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/parse-osmium.cpp:128
#82 0x000055b01f04ff2d in main (argc=<optimized out>, argv=<optimized out>)
    at /srv/nominatim/Nominatim-3.2.0/osm2pgsql/osm2pgsql.cpp:86

I wonder if anything is wrong with this run, or such anomalies are possible.

asked 06 Jun, 10:27

Leon%20Manukyan's gravatar image

Leon Manukyan
3112
accept rate: 0%


permanent link

answered 06 Jun, 10:36

SimonPoole's gravatar image

SimonPoole ♦
38.6k13287612
accept rate: 19%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×556
×211
×49
×31
×26

question asked: 06 Jun, 10:27

question was seen: 248 times

last updated: 06 Jun, 10:36

powered by OSQA