Skip to content

Bdd select step #2435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
joto merged 3 commits into from
Nov 29, 2025
Merged

Bdd select step #2435

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3913c6d
bdd: use a SQL statement template for index checks
lonvia Nov 29, 2025
7d69ce8
bdd: remove needs include index check
lonvia Nov 29, 2025
69739e8
bdd: replace remaining SELECT steps with SQL templates
lonvia Nov 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 1 addition & 7 deletions tests/bdd/environment.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ def before_scenario(context, scenario):
context.geometry_factory = GeometryFactory()
context.osm2pgsql_replication.ReplicationServer = ReplicationServerMock()
context.urlrequest_responses = {}
context.sql_statements = {}

def _mock_urlopen(request):
if not request.full_url in context.urlrequest_responses:
Expand Down Expand Up @@ -153,10 +154,3 @@ def test_db(context, **kwargs):
def working_directory(context, **kwargs):
with tempfile.TemporaryDirectory() as tmpdir:
yield Path(tmpdir)


def before_tag(context, tag):
if tag == 'needs-pg-index-includes':
if context.config.userdata['PG_VERSION'] < 110000:
context.scenario.skip("No index includes in PostgreSQL < 11")

34 changes: 21 additions & 13 deletions tests/bdd/flex/delete-callbacks.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,21 +27,29 @@ Feature: Test for delete callbacks

Given the input file '008-ch.osc.gz'

Given the SQL statement grouped_counts
"""
SELECT osm_type,
count(*),
sum(extra) AS sum_extra,
sum(osm_id) AS sum_osm_id
FROM change
GROUP BY osm_type
"""

Scenario: Delete callbacks are called
When running osm2pgsql flex with parameters
| --slim | -a |

Then SELECT osm_type, count(*), sum(extra) FROM change GROUP BY osm_type
| osm_type | count | sum |
| N | 16773 | 16779 |
| W | 4 | 9 |
| R | 1 | 3 |
Then statement grouped_counts returns exactly
| osm_type | count | sum_extra |
| N | 16773 | 16779 |
| W | 4 | 9 |
| R | 1 | 3 |

When running osm2pgsql flex with parameters
| --slim | -a |

Then SELECT osm_type, count(*), sum(osm_id) FROM change GROUP BY osm_type
| osm_type | count | sum |
Then statement grouped_counts returns exactly
| osm_type | count | sum_osm_id |
| N | 16773 | 37856781001834 |
| W | 4 | 350933407 |
| R | 1 | 2871571 |
Expand Down Expand Up @@ -72,8 +80,8 @@ Feature: Test for delete callbacks
"""
When running osm2pgsql flex with parameters
| --slim | -a |
Then SELECT osm_type, count(*), sum(extra) FROM change GROUP BY osm_type
| osm_type | count | sum |
Then statement grouped_counts returns exactly
| osm_type | count | sum_extra |
| N | 16773 | 16773 |
| W | 4 | 4 |
| R | 1 | 1 |
Expand Down Expand Up @@ -105,8 +113,8 @@ Feature: Test for delete callbacks
"""
When running osm2pgsql flex with parameters
| --slim | -a |
Then SELECT osm_type, count(*), sum(extra) FROM change GROUP BY osm_type
| osm_type | count | sum |
Then statement grouped_counts returns exactly
| osm_type | count | sum_extra |
| N | 16773 | 16773|
| W | 4 | 4|
| R | 1 | 1|
125 changes: 61 additions & 64 deletions tests/bdd/flex/lua-index-definitions.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
Feature: Index definitions in Lua file

Background:
Given the SQL statement mytable_indexes
"""
SELECT indexdef, indisprimary as is_primary
FROM pg_catalog.pg_index, pg_catalog.pg_indexes
WHERE schemaname = 'public'
AND tablename = 'mytable'
AND indrelid = tablename::regclass
AND indexrelid = indexname::regclass
"""

Scenario: Indexes field in table definition must be an array
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
And the lua style
Expand Down Expand Up @@ -36,9 +47,9 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING gist (geom)%'
| schemaname | tablename |
| public | mytable |
Then statement mytable_indexes returns
| indexdef@substr |
| USING gist (geom) |

Scenario: Empty indexes field in table definition gets you no index
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand All @@ -56,8 +67,8 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable'
| schemaname | tablename |
Then statement mytable_indexes returns exactly
| indexdef |

Scenario: Explicitly setting an index column works
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand All @@ -77,9 +88,9 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING btree (name)%'
| schemaname | tablename |
| public | mytable |
Then statement mytable_indexes returns
| indexdef@substr |
| USING btree (name) |

Scenario: Explicitly setting multiple indexes
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand All @@ -101,15 +112,11 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING btree (name)%'
| schemaname | tablename |
| public | mytable |
And SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING gist (geom)%'
| schemaname | tablename |
| public | mytable |
And SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING btree (name, tags)%'
| schemaname | tablename |
| public | mytable |
Then statement mytable_indexes returns exactly
| indexdef@substr |
| USING btree (name) |
| USING gist (geom) |
| USING btree (name, tags) |

Scenario: Method can not be missing
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand Down Expand Up @@ -244,11 +251,10 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING btree (lower(name))%'
| schemaname | tablename |
| public | mytable |
Then statement mytable_indexes returns
| indexdef@substr |
| USING btree (lower(name)) |

@needs-pg-index-includes
Scenario: Include field must be a string or array
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
And the lua style
Expand All @@ -272,7 +278,6 @@ Feature: Index definitions in Lua file
The 'include' field in an index definition must contain a string or an array.
"""

@needs-pg-index-includes
Scenario: Include field must contain a valid column
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
And the lua style
Expand All @@ -296,7 +301,6 @@ Feature: Index definitions in Lua file
Unknown column 'foo' in table 'mytable'.
"""

@needs-pg-index-includes
Scenario: Include field works with string
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
And the lua style
Expand All @@ -315,11 +319,10 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING btree (name)%' AND indexdef LIKE '%INCLUDE (tags)%'
| schemaname | tablename |
| public | mytable |
Then statement mytable_indexes returns
| indexdef@substr |
| USING btree (name) INCLUDE (tags) |

@needs-pg-index-includes
Scenario: Include field works with array
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
And the lua style
Expand All @@ -338,9 +341,9 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING btree (name)%' AND indexdef LIKE '%INCLUDE (tags)%'
| schemaname | tablename |
| public | mytable |
Then statement mytable_indexes returns
| indexdef@substr |
| USING btree (name) INCLUDE (tags) |

Scenario: Tablespace needs a string
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand Down Expand Up @@ -383,9 +386,9 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING btree (name)%'
| schemaname | tablename |
| public | mytable |
Then statement mytable_indexes returns
| indexdef@substr |
| USING btree (name) |

Scenario: Unique needs a boolean
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand Down Expand Up @@ -428,9 +431,9 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then SELECT schemaname, tablename FROM pg_catalog.pg_indexes WHERE tablename = 'mytable' AND indexdef LIKE '%USING btree (name)%' AND indexdef LIKE '%UNIQUE%'
| schemaname | tablename |
| public | mytable |
Then statement mytable_indexes returns
| indexdef@fullmatch |
| .*UNIQUE.*USING btree \(name\).* |

Scenario: Where condition needs a string
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand Down Expand Up @@ -473,9 +476,9 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then table pg_catalog.pg_indexes contains
| schemaname | tablename | indexdef@fullmatch |
| public | mytable | .*USING btree \(name\).*WHERE \(name = lower\(name\)\).* |
Then statement mytable_indexes returns
| indexdef@substr |
| USING btree (name) WHERE (name = lower(name)) |


Scenario: Don't create id index if the configuration doesn't mention it
Expand All @@ -493,9 +496,9 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then table pg_catalog.pg_indexes doesn't contain
| schemaname | tablename | indexname@fullmatch |
| public | mytable | .*node_id.* |
Then statement mytable_indexes returns exactly
| indexdef@substr |
| USING gist (geom) |

Scenario: Don't create id index if the configuration doesn't says so
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand All @@ -512,9 +515,9 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then table pg_catalog.pg_indexes doesn't contain
| schemaname | tablename | indexname@fullmatch |
| public | mytable | .*node_id.* |
Then statement mytable_indexes returns exactly
| indexdef@substr |
| USING gist (geom) |

Scenario: Always create id index if the configuration says so
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
Expand All @@ -531,16 +534,16 @@ Feature: Index definitions in Lua file
})
"""
When running osm2pgsql flex
Then table pg_catalog.pg_indexes contains
| schemaname | tablename | indexname@fullmatch |
| public | mytable | .*node_id.* |
Then statement mytable_indexes returns
| indexdef@substr |
| USING btree (node_id) |

Scenario: Create a unique id index when requested
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
And the lua style
"""
local t = osm2pgsql.define_table({
name = 'foo',
name = 'mytable',
ids = { type = 'node', id_column = 'node_id', create_index = 'unique' },
columns = {}
})
Expand All @@ -550,20 +553,17 @@ Feature: Index definitions in Lua file
end
"""
When running osm2pgsql flex
Then table foo has 1562 rows
Then table pg_catalog.pg_indexes contains
| tablename | indexdef@fullmatch |
| foo | CREATE UNIQUE INDEX .* USING .*\(node_id\) |
And SELECT count(*) FROM pg_catalog.pg_index WHERE indrelid = 'foo'::regclass and indisprimary
| count |
| 0 |
Then table mytable has 1562 rows
Then statement mytable_indexes returns
| indexdef@fullmatch | is_primary |
| CREATE UNIQUE INDEX .* USING .*\(node_id\).* | False |

Scenario: Create a primary key id index when requested
Given the input file 'liechtenstein-2013-08-03.osm.pbf'
And the lua style
"""
local t = osm2pgsql.define_table({
name = 'foo',
name = 'mytable',
ids = { type = 'node', id_column = 'node_id', create_index = 'primary_key' },
columns = {}
})
Expand All @@ -573,10 +573,7 @@ Feature: Index definitions in Lua file
end
"""
When running osm2pgsql flex
Then table foo has 1562 rows
Then table pg_catalog.pg_indexes contains
| tablename | indexdef@fullmatch |
| foo | CREATE UNIQUE INDEX .* USING .*\(node_id\) |
And SELECT count(*) FROM pg_catalog.pg_index WHERE indrelid = 'foo'::regclass and indisprimary
| count |
| 1 |
Then table mytable has 1562 rows
Then statement mytable_indexes returns
| indexdef@fullmatch | is_primary |
| CREATE UNIQUE INDEX .* USING .*\(node_id\) | True |
Loading