Skip to content

NDB: Exception when sorting by KeyProperty when keys have both int and string IDs #1007

@ndenny

Description

@ndenny

Environment details

Python 3.12:

google-cloud-core==2.4.1
google-cloud-datastore==2.20.1
google-cloud-ndb==2.3.2

Steps to reproduce

  1. Query an NDB model that uses a KeyProperty in the ordering and the keys stored mix string IDs with Integer IDs.

Code example

(Rough - admittedly I didn't run this)

from google.cloud import ndb

class SomeModel(ndb.Model):
  key_val = ndb.KeyProperty()

SomeModel(key_val=ndb.Key("ModelName", 1)).put()
SomeModel(key_val=ndb.Key("ModelName", "a")).put()

query = SomeModel.query().order(SomeModel.key_val)
query.fetch()

Stack trace

File "/app/models/example.py", line 10, in main:
     query.fetch()
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/query.py", line 1201, in wrapper
     return wrapped(self, *dummy_args, _options=query_options)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/utils.py", line 118, in wrapper
     return wrapped(*args, **new_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/utils.py", line 150, in positional_wrapper
     return wrapped(*args, **kwds)
            ^^^^^^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/query.py", line 1743, in fetch
     return self.fetch_async(_options=kwargs["_options"]).result()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 210, in result
     self.check_success()
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 157, in check_success
     raise self._exception
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 319, in _advance_tasklet
     yielded = self.generator.throw(type(error), error, traceback)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 116, in fetch
     while (yield results.has_next_async()):
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/tasklets.py", line 323, in _advance_tasklet
     yielded = self.generator.send(send_value)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 677, in has_next_async
     if value < min_value:
        ^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 778, in __lt__
     return self._compare(other) == -1
            ^^^^^^^^^^^^^^^^^^^^
   File "/opt/venv/lib/python3.12/site-packages/google/cloud/ndb/_datastore_query.py", line 834, in _compare
     if this_value < other_value:
        ^^^^^^^^^^^^^^^^^^^^^^^^
 TypeError: '<' not supported between instances of 'str' and 'int'

This appears to be related to #630 - I see the unit tests added didn't test having both strings and ints as key IDs in the same query result.

Perhaps this is not an expected use of NDB - I wouldn't have seen this except for some very old data in our datastore! But I'm fairly sure our old Python2 code made a similar query and didn't have this problem.

I can work around this issue for now, so I don't consider this be of high importance, but thought I'd log it with you. Feel free to go back to your holidays :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: datastoreIssues related to the googleapis/python-ndb API.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions