About GPU memory consumption

[Vilour]

Thank you for your great work!

If I want to run a dataset with N>20,000 images, how much GPU VRAM should I prepare?

[hyhan0118]

The only dense matrix you need to keep on-GPU is the $N\times N$ Q matrix, which alone consumes about 3 GB of memory. Accounting for the additional sparse structures, I’d recommend having at least 5 GB of GPU RAM to run comfortably.

[Vilour]

The only dense matrix you need to keep on-GPU is the N × N Q matrix, which alone consumes about 3 GB of memory. Accounting for the additional sparse structures, I’d recommend having at least 5 GB of GPU RAM to run comfortably.

I tried with my own dataset and it seems requiring very large RAM to store matrix:

Number of connected components: 1
M: 1922636, N: 18810
Traceback (most recent call last):
File "/home/SS/XM-code/4_test_unidepth.py", line 298, in
create_matrix(weights, edges, landmarks, output_path)
File "/home/SS/XM-code/utils/creatematrix.py", line 77, in create_matrix
V2 = np.zeros((3 * N, M))
numpy._core._exceptions._ArrayMemoryError: Unable to allocate 808. GiB for an array with shape (56430, 1922636) and data type float64

Is it normal?

[hyhan0118]

Current math formulation needs to generate the $Q$ matrix, this part is highly relevant to the number of landmarks. You have 2 million landmarks which may be too much for current solver. I suggest the following ways:

1. You down-sample the landmarks to make it smaller
2. I turn the matrix to sparse and update a separate function
3. I switch to another formulation in Sim-sync, which do not need to store large matrix but cannot solve out landmarks (only camera poses)

Could you send the dataset to me so I can test it? My email is hyhan@seas.harvard.edu.

[Vilour]

Current math formulation needs to generate the Q matrix, this part is highly relevant to the number of landmarks. You have 2 million landmarks which may be too much for current solver. I suggest the following ways:

1. You down-sample the landmarks to make it smaller

2. I turn the matrix to sparse and update a separate function

3. I switch to another formulation in [Sim-sync](https://arxiv.org/abs/2309.05184), which do not need to store large matrix but cannot solve out landmarks (only camera poses)

Could you send the dataset to me so I can test it? My email is hyhan@seas.harvard.edu.

Thank you! What kind of data do you need? Is it enough if I only send you the dataset.db?

[hyhan0118]

I think either the whole image datasets or the output from glomap, everything should in your output/ folder.

[Vilour]

I think either the whole image datasets or the output from glomap, everything should in your output/ folder.

I have sent the dataset.db and output folder through google drive, please have a check!

[hyhan0118]

I see it, let me give it a try.

[xiemeilong]

Is there any progress on this issue? @hyhan0118

[hyhan0118]

I think you will need a big CPU memory to run large dataset, I am still working on another formulation that does not require big RAM.