Docker build fails at fetch-examples: Cannot find module '../models/user

[nbogie]

p5.js version

n/a current develop branch #2c52f25bec4a5acddca2a04c290f59df8846e9d3

What is your operating system?

Mac OS

Web browser and version

n/a

Actual Behavior

On a new clone and fresh (docker-based) install and full build of the project, the build step goes fine but the fetch-examples step fails.

docker-compose -f docker-compose-development.yml build

Output:

[omitted - no problems.  will attach as file]
...
p5webeditor-again-app  Built

(exit code 0)

Then it fails on the following:

docker-compose -f docker-compose-development.yml run --rm app npm run fetch-examples

Output:

✔ Container p5webeditor-again-mongo-1  Running                                                                                                        0.0s

> p5.js-web-editor@2.18.4 fetch-examples
> cross-env NODE_ENV=development node ./server/scripts/fetch-examples.js

node:internal/modules/cjs/loader:1143
  throw err;
  ^

Error: Cannot find module '../models/user'
Require stack:
- /usr/src/app/server/scripts/examples.js
- /usr/src/app/server/scripts/fetch-examples.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1140:15)
    at Function.Module._load (node:internal/modules/cjs/loader:981:27)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/usr/src/app/server/scripts/examples.js:7:1)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._compile (/usr/src/app/node_modules/pirates/lib/index.js:117:24)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Object.newLoader [as .js] (/usr/src/app/node_modules/pirates/lib/index.js:121:7)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1019:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/usr/src/app/server/scripts/fetch-examples.js:8:1)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1019:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12)
    at node:internal/main/run_main_module:28:49 {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/usr/src/app/server/scripts/examples.js',
    '/usr/src/app/server/scripts/fetch-examples.js'
  ]
}

Node.js v18.20.8

Extra info:

At a guess it looks like the TS is not being transpiled, or not picked up, despite the PR #3696 which I see has been merged into develop.

After manually adding in a .js version of /server/models/user.ts (and a couple of other dependencies) into the same dir, I'm able to re-run this fetch-examples step and see it complete without problem. (Subsequently, I could bring the server up and use the editor successfully, too.)
This suggests my system setup isn't wildly wrong, at least.

I found #3694 (closed) but the errors are different, so not a duplicate, I think? Should I just add to that issue?

Expected Behavior

Expected to fork, clone the develop branch of the web editor, and run through the docker-based install+build instructions and get the server running without the build process failing.

Steps to reproduce

1. install docker
2. install docker desktop
3. clone develop branch of this repo (commit #2c52f25bec4a5acddca2a04c290f59df8846e9d3) and cd in
4. run docker-compose -f docker-compose-development.yml build
5. create oauth app on github
6. add github credentials to .env, per the guide
7. rundocker-compose -f docker-compose-development.yml run --rm app npm run fetch-examples

[welcome]

Welcome! 👋 Thanks for opening your first issue here! And to ensure the community is able to respond to your issue, be sure to follow the issue template if you haven't already.

[nbogie]

Related to #3694 and #3696.

[nbogie]

(It's my first time building the web editor project, so it's very possible I'm screwing up / missing an instruction step. Sorry if that's the case.)

[nbogie]

3714-docker-output-of-first-build-step.txt
I attached the omitted output from docker of the first build step docker-compose -f docker-compose-development.yml build

(Nothing useful in there, I think - it doesn't indicate if build:examples actually ran, for example)

[clairep94]

Copying in discussion from discord between me & nbogie: https://discord.com/channels/836700474425475088/1007853564363817060/1435377191473905795

@nbogie

@Claire here's a screen recording of what I see when trying to get the docker-based build up: https://www.loom.com/share/ceaae6d6612b46219c9890d4302e10c1
For issue #3714.

@clairep94

Hi @NZ0H thanks for that!
I have a suspicion it's something along the lines of:

For developers installing for the first time, they may run the fetch-examples script, which uses ./examples
in ./examples the imports dont have extensions, which is expected, but we did stuff in other parts of the app where the extension automatically resolves to either TS or JS (or tsx/jsx)

I think we'll need to do something similar in fetch-examples
I'm not that familiar with this file or how it works, so not 100% about this!

The build process is unaffected because it doesn't run fetch-examples

@NZ0H apologies for this ask, but I'm on a work laptop and not able to install the project
Could you try updating the require('@babel/register') bit in fetch-examples to the below?
require('@babel/register')({
extensions: ['.js', '.jsx', '.ts', '.tsx'],
presets: ['@babel/preset-env', '@babel/preset-typescript']
});

This is what I did in root/index.js, which auto-resolved the extensions in the server
https://github.com/processing/p5.js-web-editor/pull/3636/files#diff-e727e4bdf3657fd1d798edcd6b099d6e092f8573cba266154583a746bba0f346R10-R11

@nbogie

Brilliant, yes, that seems to fix fetch-examples, thank you!
And I have checked that I can then go on and start and use the server fine a little bit (and load and play an example from File:Examples, too)

=> @nbogie to make PR for above fix

[nbogie]

I did prep the PR as promised, but Claire finalised one from her side as a mongodb version error was causing npm test to fail on my (docker-based) setup.

It is failing around the same model, interestingly:

FAIL   server server/models/__test__/user.test.ts

Requested Version "7.0.0" is not available for "Debian 12"! Available Versions: ">=7.0.3"
Mongodb does not provide binaries for versions before 7.0.3 for Debian 12+ and also cannot be mapped to a previous Debian release

This test failure happens both before and after the application of the transpilation fix.

Anyway, I can revisit once her PR is merged into dev.

[raclim]

Thanks for the thorough documentation on this @nbogie @clairep94! Following up that it's just been merged in.

[nbogie]

Thanks, @raclim !

(I can confirm that docker build + fetch-examples seems to run fine for me now after refreshing develop branch!)