docs: update documentation and split docs into separate files

Author: matmoncon

.github/workflows/ci.yaml

@@ -17,7 +17,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.10.6, 3.11]
+        python-version: ["3.10", "3.11", "3.12"]
 
     steps:
       - uses: actions/checkout@v3
@@ -61,7 +61,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.10.6, 3.11]
+        python-version: ["3.10", "3.11", "3.12"]
 
     steps:
       - uses: actions/checkout@v3
@@ -112,7 +112,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.10.6, 3.11]
+        python-version: ["3.10", "3.11", "3.12"]
         pydantic-version: ["1.10.9", "^2"]
 
     steps:

.vscode/settings.json

@@ -8,8 +8,8 @@
   "python.testing.pytestEnabled": true,
   "markdownlint.config": {
     "MD024": false,
-    "MD033": false,
-    "MD039": false
+    "MD039": false,
+    "MD041": false
   },
   "files.exclude": {
     "**/.git": true,

README.md

@@ -0,0 +1,18 @@
+## Basic concepts
+
+As you might have guessed by now, `pyneo4j-ogm` is a library that allows you to interact with a Neo4j database using Python. It is designed to make your life as simple as possible, while still providing the most common operations and some more advanced features.
+
+But first, how does this even work!?! Well, the basic concept boils down to the following:
+
+- You define your models that represent your nodes and relationships inside the graph.
+- You use these models to do all sorts of things with your data.
+
+Of course, there is a lot more to it than that, but this is the basic idea. So let's take a closer look at the different parts of `pyneo4j-ogm` and how to use them.
+
+> **Note:** All of the examples in this documentation assume that you have already connected to a database and registered your models with the client like shown in the [`quickstart guide`](https://github.com/groc-prog/pyneo4j-ogm/blob/develop?tab=readme-ov-file#-quickstart). The models used in the following examples will build upon the ones defined there. If you are new to [`Neo4j`](https://neo4j.com/docs/) or [`Cypher`](https://neo4j.com/docs/cypher-manual/current/) in general, you should get a basic understanding of how to use them before continuing.
+
+### A note on Pydantic version support
+
+As of version [`v0.3.0`](https://github.com/groc-prog/pyneo4j-ogm/blob/develop/CHANGELOG.md#whats-changed-in-v030-2023-11-30), pyneo4j-ogm now supports both `Pydantic 1.10+ and 2+`. All core features of pydantic should work, meaning full support for model serialization, validation and schema generation.
+
+Should you find any issues or run into any problems, feel free to open a issue!

docs/Concept.md

@@ -0,0 +1,171 @@
+## Database client
+
+This is where the magic happens! The `Pyneo4jClient` is the main entry point for interacting with the database. It handles all the heavy lifting for you and your models. Because of this, we have to always have at least one client initialized before doing anything else.
+
+### Connecting to the database
+
+Before you can run any queries, you have to connect to a database. This is done by calling the `connect()` method of the `Pyneo4jClient` instance. The `connect()` method takes a few arguments:
+
+- `uri`: The connection URI to the database.
+- `skip_constraints`: Whether the client should skip creating any constraints defined on models when registering them. Defaults to `False`.
+- `skip_indexes`: Whether the client should skip creating any indexes defined on models when registering them. Defaults to `False`.
+- `*args`: Additional arguments that are passed directly to Neo4j's `AsyncDriver.driver()` method.
+- `**kwargs`: Additional keyword arguments that are passed directly to Neo4j's `AsyncDriver.driver()` method.
+
+```python
+from pyneo4j_ogm import Pyneo4jClient
+
+client = Pyneo4jClient()
+await client.connect(uri="<connection-uri-to-database>", auth=("<username>", "<password>"), max_connection_pool_size=10, ...)
+
+## Or chained right after the instantiation of the class
+client = await Pyneo4jClient().connect(uri="<connection-uri-to-database>", auth=("<username>", "<password>"), max_connection_pool_size=10, ...)
+```
+
+After connecting the client, you will be able to run any cypher queries against the database. Should you try to run a query without connecting to a database first (it happens to the best of us), you will get a `NotConnectedToDatabase` exception.
+
+### Closing an existing connection
+
+Connections can explicitly be closed by calling the `close()` method. This will close the connection to the database and free up any resources used by the client. Remember to always close your connections when you are done with them!
+
+```python
+## Do some heavy-duty work...
+
+## Finally done, so we close the connection to the database.
+await client.close()
+```
+
+Once you closed the client, it will be seen as `disconnected` and if you try to run any further queries with it, you will get a `NotConnectedToDatabase` exception
+
+### Registering models
+
+Models are a core feature of pyneo4j-ogm, and therefore you probably want to use some. But to work with them, they have to be registered with the client by calling the `register_models()` method and passing in your models as a list:
+
+```python
+## Create a new client instance and connect ...
+
+await client.register_models([Developer, Coffee, Consumed])
+```
+
+This is a crucial step, because if you don't register your models with the client, you won't be able to work with them in any way. Should you try to work with a model that has not been registered, you will get a `UnregisteredModel` exception. This exception also gets raised if a database model defines a relationship-property with other (unregistered) models as a target or relationship model and then runs a query with said relationship-property.
+
+If you have defined any indexes or constraints on your models, they will be created automatically when registering them. You can prevent this behavior by passing `skip_constraints=True` or `skip_indexes=True` to the `connect()` method. If you do this, you will have to create the indexes and constraints yourself.
+
+> **Note**: If you don't register your models with the client, you will still be able to run cypher queries directly with the client, but you will `lose automatic model resolution` from queries. This means that, instead of resolved models, the raw Neo4j query results are returned.
+
+### Executing Cypher queries
+
+Models aren't the only things capable of running queries. The client can also be used to run queries, with some additional functionality to make your life easier.
+
+Node- and RelationshipModels provide many methods for commonly used cypher queries, but sometimes you might want to execute a custom cypher with more complex logic. For this purpose, the client instance provides a `cypher()` method that allows you to execute custom cypher queries. The `cypher()` method takes three arguments:
+
+- `query`: The cypher query to execute.
+- `parameters`: A dictionary containing the parameters to pass to the query.
+- `resolve_models`: Whether the client should try to resolve the models from the query results. Defaults to `True`.
+
+This method will always return a tuple containing a list of results and a list of variables returned by the query. Internally, the client uses the `.values()` method of the Neo4j driver to get the results of the query.
+
+> **Note:** If no models have been registered with the client and resolve_models is set to True, the client will not raise any exceptions but rather return the raw query results.
+
+Here is an example of how to execute a custom cypher query:
+
+```python
+results, meta = await client.cypher(
+  query="CREATE (d:Developer {uid: '553ac2c9-7b2d-404e-8271-40426ae80de0', name: 'John', age: 25}) RETURN d.name as developer_name, d.age",
+  parameters={"name": "John Doe"},
+  resolve_models=False,  ## Explicitly disable model resolution
+)
+
+print(results)  ## [["John", 25]]
+print(meta)  ## ["developer_name", "d.age"]
+```
+
+### Batching cypher queries
+
+We provide an easy way to batch multiple database queries together, regardless of whether you are using the client directly or via a model method. To do this you can use the `batch()` method, which has to be called with a asynchronous context manager like in the following example:
+
+```python
+async with client.batch():
+  ## All queries executed inside the context manager will be batched into a single transaction
+  ## and executed once the context manager exits. If any of the queries fail, the whole transaction
+  ## will be rolled back.
+  await client.cypher(
+    query="CREATE (d:Developer {uid: $uid, name: $name, age: $age})",
+    parameters={"uid": "553ac2c9-7b2d-404e-8271-40426ae80de0", "name": "John Doe", "age": 25},
+  )
+  await client.cypher(
+    query="CREATE (c:Coffee {flavour: $flavour, milk: $milk, sugar: $sugar})",
+    parameters={"flavour": "Espresso", "milk": False, "sugar": False},
+  )
+
+  ## Model queries also can be batched together without any extra work!
+  coffee = await Coffee(flavour="Americano", milk=False, sugar=False).create()
+```
+
+You can batch anything that runs a query, be that a model method, a custom query or a relationship-property method. If any of the queries fail, the whole transaction will be rolled back and an exception will be raised.
+
+### Using bookmarks (Enterprise Edition only)
+
+If you are using the Enterprise Edition of Neo4j, you can use bookmarks to keep track of the last transaction that has been committed. The client provides a `last_bookmarks` property that allows you to get the bookmarks from the last session. These bookmarks can be used in combination with the `use_bookmarks()` method. Like the `batch()` method, the `use_bookmarks()` method has to be called with a context manager. All queries run inside the context manager will use the bookmarks passed to the `use_bookmarks()` method. Here is an example of how to use bookmarks:
+
+```python
+## Create a new node and get the bookmarks from the last session
+await client.cypher("CREATE (d:Developer {name: 'John Doe', age: 25})")
+bookmarks = client.last_bookmarks
+
+## Create another node, but this time don't get the bookmark
+## When we use the bookmarks from the last session, this node will not be visible
+await client.cypher("CREATE (c:Coffee {flavour: 'Espresso', milk: False, sugar: False})")
+
+with client.use_bookmarks(bookmarks=bookmarks):
+  ## All queries executed inside the context manager will use the bookmarks
+  ## passed to the `use_bookmarks()` method.
+
+  ## Here we will only see the node created in the first query
+  results, meta = await client.cypher("MATCH (n) RETURN n")
+
+  ## Model queries also can be batched together without any extra work!
+  ## This will return no results, since the coffee node was created after
+  ## the bookmarks were taken.
+  coffee = await Coffee.find_many()
+  print(coffee)  ## []
+```
+
+### Manual indexing and constraints
+
+Most of the time, the creation of indexes/constraints will be handled by the models themselves. But it can still be handy to have a simple way of creating new ones. This is where the `create_lookup_index()`, `create_range_index`, `create_text_index`, `create_point_index` and `create_uniqueness_constraint()` methods come in.
+
+First, let's take a look at how to create a custom index in the database. The `create_range_index`, `create_text_index` and `create_point_index` methods take a few arguments:
+
+- `name`: The name of the index to create (Make sure this is unique!).
+- `entity_type`: The entity type the index is created for. Can be either **EntityType.NODE** or **EntityType.RELATIONSHIP**.
+- `properties`: A list of properties to create the index for.
+- `labels_or_type`: The node labels or relationship type the index is created for.
+
+The `create_lookup_index()` takes the same arguments, except for the `labels_or_type` and `properties` arguments.
+
+The `create_uniqueness_constraint()` method also takes similar arguments.
+
+- `name`: The name of the constraint to create.
+- `entity_type`: The entity type the constraint is created for. Can be either **EntityType.NODE** or **EntityType.RELATIONSHIP**.
+- `properties`: A list of properties to create the constraint for.
+- `labels_or_type`: The node labels or relationship type the constraint is created for.
+
+Here is an example of how to use the methods:
+
+```python
+## Creates a `RANGE` index for a `Coffee's` `sugar` and `flavour` properties
+await client.create_range_index("hot_beverage_index", EntityType.NODE, ["sugar", "flavour"], ["Beverage", "Hot"])
+
+## Creates a UNIQUENESS constraint for a `Developer's` `uid` property
+await client.create_uniqueness_constraint("developer_constraint", EntityType.NODE, ["uid"], ["Developer"])
+```
+
+### Client utilities
+
+The client also provides some additional utility methods, which mostly exist for convenience when writing tests or setting up environments:
+
+- `is_connected()`: Returns whether the client is currently connected to a database.
+- `drop_nodes()`: Drops all nodes from the database.
+- `drop_constraints()`: Drops all constraints from the database.
+- `drop_indexes()`: Drops all indexes from the database.

docs/DatabaseClient.md

@@ -0,0 +1,3 @@
+## Logging
+
+You can control the log level and whether to log to the console or not by setting the `PYNEO4J_OGM_LOG_LEVEL` and `PYNEO4J_OGM_ENABLE_LOGGING` as environment variables. The available levels are the same as provided by the build-in `logging` module. The default log level is `WARNING` and logging to the console is enabled by default.

docs/Logging.md

@@ -0,0 +1,88 @@
+## Migrations
+
+As of version [`v0.5.0`](https://github.com/groc-prog/pyneo4j-ogm/blob/develop/CHANGELOG.md#whats-changed-in-v050-2024-02-06), pyneo4j-ogm supports migrations using a built-in migration tool. The migration tool is basic but flexibly, which should cover most use-cases.
+
+### Initializing migrations for your project
+
+To initialize migrations for your project, you can use the `poetry run pyneo4j_ogm init` command. This will create a `migrations` directory at the given path (which defaults to `./migrations`), which will contain all your migration files.
+
+```bash
+poetry run pyneo4j_ogm init --migration-dir ./my/custom/migration/path
+```
+
+### Creating a new migration
+
+To create a new migration, you can use the `poetry run pyneo4j_ogm create` command. This will create a new migration file inside the `migrations` directory. The migration file will contain a `up` and `down` function, which you can use to define your migration.
+
+```bash
+poetry run pyneo4j_ogm create my_first_migration
+```
+
+Both the `up` and `down` functions will receive the client used during the migration as their only arguments. This makes the migrations pretty flexible, since you can not only use the client to execute queries, but also register models on it and use them to execute methods.
+
+> **Note**: When using models inside the migration, you have to make sure that the model used implements the same data structure as the data inside the graph. Otherwise you might run into validation issues.
+
+```python
+"""
+Auto-generated migration file {name}. Do not
+rename this file or the `up` and `down` functions.
+"""
+from pyneo4j_ogm import Pyneo4jClient
+
+
+async def up(client: Pyneo4jClient) -> None:
+    """
+    Write your `UP migration` here.
+    """
+    await client.cypher("CREATE (n:Node {name: 'John'})")
+
+
+async def down(client: Pyneo4jClient) -> None:
+    """
+    Write your `DOWN migration` here.
+    """
+    await client.cypher("MATCH (n:Node {name: 'John'}) DELETE n")
+```
+
+### Running migrations
+
+To run the migrations, you can use the `up` or `down` commands. The `up` command will run all migrations that have not been run yet, while the `down` command will run all migrations in reverse order.
+
+Both commands support a `--up-count` or `--down-count` argument, which can be used to limit the number of migrations to run. By default, the `up` command will run `all pending migration` and the `down` command will roll back the `last migration`.
+
+```bash
+poetry run pyneo4j_ogm up --up-count 3
+poetry run pyneo4j_ogm down --down-count 2
+```
+
+### Listing migrations
+
+The current state of all migrations can be viewed anytime using the `status` command. This will show you all migrations that have been run and all migrations that are pending.
+
+```bash
+poetry run pyneo4j_ogm status
+
+## Output
+┌─────────────────────────────────────────┬─────────────────────┐
+│ Migration                               │ Applied At          │
+├─────────────────────────────────────────┼─────────────────────┤
+│ 20160608155948-my_awesome_migration     │ 2022-03-04 15:40:22 │
+│ 20160608155948-my_fixed_migration       │ 2022-03-04 15:41:13 │
+│ 20160608155948-final_fix_i_swear        │ PENDING             │
+└─────────────────────────────────────────┴─────────────────────┘
+```
+
+### Programmatic usage
+
+The migration tool can also be used programmatically. This can be useful if you want to run migrations inside your application or if you want to integrate the migration tool into your own CLI.
+
+```python
+import asyncio
+from pyneo4j_ogm.migrations import create, down, init, status, up
+
+## Call with same arguments as you would with cli
+init(migration_dir="./my/custom/migration/path")
+
+create("my_first_migration")
+asyncio.run(up())
+```