SOLVED

Following a comment below saying the script works for another user, I created a greatly simplified Overpass query to pull the missing data and it works fine. This tells me that the nodes are missing from the source query results so there's actually nothing wrong with my LUA script.

Thanks for the replies, if I can't work out the issue with the overpass query I'll open another question. Hopefully I won't have to :)

I've been working on a LUA import script for osm2pgsql and once I had it all working I checked the database in qgis and found that some ways are missing.

I can see them in the source XML file (from overpass), with all their node references but osm2pgsql does not import them.

I've created a small test script to try and visualise what is happening

inspect = require('inspect')

local tables = {}

tables.way_test = osm2pgsql.define_table({
    name = 'way_test',
    ids = { type = 'way', id_column = "id"},
    columns = {
        { column = 'name', type = 'text' },
        { column = 'geom', type = 'geometry' },
    }
})

function osm2pgsql.process_way(object)
    if object.id == 987525760 then
        print(inspect(object))
        print("line", object.as_linestring())
        print("poly", object.as_polygon())
        print("mline", object.as_multilinestring())
        print("mpoly", object.as_multipolygon())
    end

    tables.way_test:add_row({
        name = object.tags.name,
        geom = { create = 'line' },
    })
end

The above script is run with the following CLI arguments to version 1.7.0 which I compiled myself:

osm2pgsql --host=localhost --user=user --database=db --cache=40050 --flat-nodes=nodes.tmp -sWG --output=flex --style=test.lua data.pbf

This script does not add the missing lines to the test table but does tell me at all four of the as_ functions return NULL.

I've also imported the same data using the default style and the way does not appear in the lines table.

osm2pgsql --host=localhost --user=user --database=db --cache=40050 --flat-nodes=nodes.tmp -sWG --output=pgsql data.pbf

All this suggests dodgy data but the way exists and does have a geometry in OSM itself https://www.openstreetmap.org/way/987525760

While this is one specific way, there are a few I've spotted due to familiarity with the area. Some, like this one are missing while a few others seem to be a straight line from the first to last node without other nodes in between. Can anyone explain why this way is not being imported?

Versions:

osm2pgsql version 1.7.0 (1.7.0)
Build: RelWithDebInfo
Compiled using the following library versions:
Libosmium 2.17.3
Proj [API 6] 8.2.1
Lua 5.3.6