Consider multiple size-based releases 🔬🔍🔭📡

[bollwyvl]

RobotLab is big and likely to get bigger as more cool features need more (exotic) dependencies.

We could just as easily spin a couple different intstallers a la Miniconda/Anaconda in CI.

MicrobotLab 🔬

• for CI environments
  • you'll already have browsers, etc. hopefully
• don't even ship conda (pip usually works, right? and conda is showing up on more CIs)

RobotLabLite 🔍

• still a full working environment
• cut it down to the bone (opencv, nodejs, pandoc), and even repackage some things (jupyterlab assets)
• still ship conda so you can upgrade stuff

RobotLab 🔭

• as it is now!
  • even shipping Firefox is great!
• this is ideal for the original intent of training on non-trivial things, where the setup of some of the things have proven (really hard for people to install](https://stackoverflow.com/search?q=opencv+install).

RobotHub 📡

• ship a whole littlest jupyter hub
  • why not?

[datakurre]

My workshop for the next robot conference was accepted, which confirms that I will be attending the sprints there. Apparently there is still no installer available for Robot Framework, so I've been thinking, if I should try to fork this project and try this "multiple size-based releases" for that. For plain robot it would make sense to have minimal version with just robot framework, but also a version for getting started with selenium testing (the real reason for the installer).

[bollwyvl]

workshop for the next robot conference was accepted

Hooray!

fork this project

You know I'm glad to help, and a fork is just more repos to watch: I don't see any reason why we wouldn't start it here, and why we wouldn't want synchronized, automated releases of the whole shooting match. If somebody wants to then fork that, at some point, that's great.

We can even stop doing the kinda gross templating, and just manage the yaml generation more simply in python directly, if we start feeling confident. The CI build would take longer, but i think there's value in managing a single pipeline for all the stuff together.

minimal version with just robot framework

yup, as a CI thing, that would be great. MiniRobotConda. Even if it doesn't ship conda (which it probably should, if only for activation and administration chores) it can't get much smaller than about 70mb, but that's not so bad!

version for getting started with selenium testing

so even there i would imagine two flavors:

• WebRobotConda (just rf, rfsl, sl and maybe rfstl, if that plays out, and geckodriver)
• FireWebRobotConda (as above, but also firefox)

i'm really liking having firefox in our installer, and if we can do our little part to help more people test on firefox, i'm all for it. The only gotcha is on Linux, you do still have to have gtk3 installed, even when running --headless, but i'll still take it.

Unless we ship all the chromedrivers, we can't really ship any chromedrivers, but we could ship webdrivermanager, and indeed ship all of them pre-cached.

[datakurre]

Thanks. I can agree with your reasoning. I also prefer Firefox being bundled because now things just work. It also makes sense that whatever version or browser you use for surfing in the web should not interfere with learning Selenium testing. Finally, promoting Firefox would not hurt the diversity in browser ecosystem...

That said I cannot really focus on this before November, but will then.

[bollwyvl]

That said I cannot really focus on this before November, but will then.

I hear you, my friend. Who knows, maybe i'll be able to throw some stuff at it. What is your priority for getting something out? Maybe hack on the summary of this issue to make it clear. I guess I would lean towards the smallest functioning robot environment, just so we have a size/complexity baseline, but if the web one is more useful we could start there.

[datakurre]

I’d vote for the baseline. We already know that we can make a GB sized installer with everything :)

After the baseline, another baseline with SL, gecko and Firefox, and then documented approach to fork and extend those baselines.

Finally, at the conf it makes sense to discuss what libraries should be bundled with the default starter (for learning and getting started purposes):

[bollwyvl]

i did a little playing with this last night:

609M Sep 30 23:58 RobotLab-2019.9.1-Linux-x86_64.sh
 84M Oct  1 00:05 MiniRobot-2019.9.1-Linux-x86_64.sh
161M Oct  1 00:35 ExoRobot-2019.9.1-Linux-x86_64.sh

Mini is rf + conda. not shipping conda only saves about 2mb.
Exo is Mini, plus every optional dependency i could find in the rf codebase. The heavy hitter is definitely wx, which screenshots uses. Might as well throw in RIDE at that point...

[datakurre]

Very cool! Which screenshots feature relies on wx? Does dialogs library work? The point for a installer is to give environment that just works, so from that point of view 200M is not much nowadays.

And now that you mentioned RIDE... does it already support Python 3? Sure many would like it to be included and it’s been hard to install on many systems because of wx. (Although that probably was because for long it required very old version of that.)

[bollwyvl]

here's the full package list of `Exo` (on linux)

asn1crypto-0.24.0-py36_1003.tar.bz2
atk-2.32.0-haf93ef1_0.tar.bz2
ca-certificates-2019.9.11-hecc5488_0.tar.bz2
cairo-1.16.0-hfb77d84_1002.tar.bz2
certifi-2019.9.11-py36_0.tar.bz2
cffi-1.12.3-py36h8022711_0.tar.bz2
chardet-3.0.4-py36_1003.tar.bz2
conda-4.7.12-py36_0.tar.bz2
conda-package-handling-1.6.0-py36h516909a_0.tar.bz2
cryptography-2.7-py36h72c5cf5_0.tar.bz2
docutils-0.15.2-py36_0.tar.bz2
expat-2.2.5-he1b5a44_1003.tar.bz2
fontconfig-2.13.1-h86ecdb6_1001.tar.bz2
freetype-2.10.0-he983fc9_1.tar.bz2
fribidi-1.0.5-h516909a_1002.tar.bz2
gdk-pixbuf-2.36.12-h7a26e22_1003.tar.bz2
gettext-0.19.8.1-hc5be6a0_1002.tar.bz2
glib-2.58.3-h6f030ca_1002.tar.bz2
gobject-introspection-1.58.2-py36h5503ade_1002.tar.bz2
graphite2-1.3.13-hf484d3e_1000.tar.bz2
gst-plugins-base-1.14.5-h0935bb2_0.tar.bz2
gstreamer-1.14.5-h36ae1b5_0.tar.bz2
gtk2-2.24.32-h586f36d_1.tar.bz2
harfbuzz-2.4.0-h9f30f68_3.tar.bz2
icu-64.2-he1b5a44_1.tar.bz2
idna-2.8-py36_1000.tar.bz2
jpeg-9c-h14c3975_1001.tar.bz2
libffi-3.2.1-he1b5a44_1006.tar.bz2
libgcc-ng-9.1.0-hdf63c60_0.tar.bz2
libglu-9.0.0-hf484d3e_1000.tar.bz2
libiconv-1.15-h516909a_1005.tar.bz2
libpng-1.6.37-hed695b0_0.tar.bz2
libstdcxx-ng-9.1.0-hdf63c60_0.tar.bz2
libtiff-4.0.10-h57b8799_1003.tar.bz2
libuuid-2.32.1-h14c3975_1000.tar.bz2
libxcb-1.13-h14c3975_1002.tar.bz2
libxml2-2.9.9-hee79883_5.tar.bz2
libxslt-1.1.33-h31b3aaa_0.tar.bz2
lxml-4.4.1-py36h7ec2d77_0.tar.bz2
lz4-c-1.8.3-he1b5a44_1001.tar.bz2
ncurses-6.1-hf484d3e_1002.tar.bz2
openssl-1.1.1c-h516909a_0.tar.bz2
pango-1.42.4-ha030887_1.tar.bz2
pathlib2-2.3.5-py36_0.tar.bz2
pcre-8.41-hf484d3e_1003.tar.bz2
pip-19.2.3-py36_0.tar.bz2
pixman-0.38.0-h516909a_1003.tar.bz2
pthread-stubs-0.4-h14c3975_1001.tar.bz2
pycosat-0.6.3-py36h14c3975_1001.tar.bz2
pycparser-2.19-py36_1.tar.bz2
pyopenssl-19.0.0-py36_0.tar.bz2
pypubsub-4.0.3-py_0.tar.bz2
pysocks-1.7.1-py36_0.tar.bz2
pyte-0.8.0-py_0.tar.bz2
python-3.6.7-h357f687_1005.tar.bz2
pyyaml-5.1.2-py36h516909a_0.tar.bz2
readline-8.0-hf8c457e_0.tar.bz2
requests-2.22.0-py36_1.tar.bz2
robotframework-3.1.2-py_0.tar.bz2
ruamel_yaml-0.15.71-py36h14c3975_1000.tar.bz2
setuptools-41.2.0-py36_0.tar.bz2
six-1.12.0-py36_1000.tar.bz2
sqlite-3.29.0-hcee41ef_1.tar.bz2
tk-8.6.9-hed695b0_1003.tar.bz2
tqdm-4.36.1-py_0.tar.bz2
urllib3-1.25.6-py36_0.tar.bz2
wcwidth-0.1.7-py_1.tar.bz2
wheel-0.33.6-py36_0.tar.bz2
wxpython-4.0.6-py36h8f66242_1.tar.bz2
xorg-kbproto-1.0.7-h14c3975_1002.tar.bz2
xorg-libice-1.0.10-h516909a_0.tar.bz2
xorg-libsm-1.2.3-h84519dc_1000.tar.bz2
xorg-libx11-1.6.8-h516909a_0.tar.bz2
xorg-libxau-1.0.9-h14c3975_0.tar.bz2
xorg-libxdmcp-1.1.3-h516909a_0.tar.bz2
xorg-libxext-1.3.4-h516909a_0.tar.bz2
xorg-libxrender-0.9.10-h516909a_1002.tar.bz2
xorg-renderproto-0.11.1-h14c3975_1002.tar.bz2
xorg-xextproto-7.3.0-h14c3975_1002.tar.bz2
xorg-xproto-7.0.31-h14c3975_1007.tar.bz2
xz-5.2.4-h14c3975_1001.tar.bz2
yaml-0.1.7-h14c3975_1001.tar.bz2
zlib-1.2.11-h516909a_1006.tar.bz2
zstd-1.4.0-h3b9ef0a_0.tar.bz2

Which screenshots feature relies on wx?

Looks like wx, gtk or pillow would work. I don't think gtk is supported on windows, but pillow is handy, anyhow...

Does dialogs library work?

if all it takes is tk! haven't tested.

RIDE... does it already support Python 3?

looks like they test with 3.6. and don't pin wx. I haven't tried packaging it in a while... couldn't get hidpi to work correctly..

[bollwyvl]

No I didn't put selenium in one of the new ones, but I do think shipping Firefox/geckodriver with it is probably the right thing to do, if it should Just Work after downloading (e.g in CI). Do you think wx/ride are worth the size hit for the web excursion? Or just also add it to Exo? You _wouldn't_ actually use wx in CI. Also what about SSHLibrary? It's first party, solid as hell, relatively light, super useful, and makes it easy to test iot stuff, as well as vms d cloud stuff... But you kinda have to know what you're doing because passwords. Put in Exo? WhiteRobot? I've still not made it work on win10, and don't like how it's packages. But sounds so good! SikuliRobot: I always consider it the Last Argument of Kings of test execution. Hard to maintain, but can probably do the thing. Packaging Java stuff is no problem, though jython probably is or will be soon, if you expect to be able to run anything else after 89 days! The whole *Robot branding thing is a bit silly, of course, and wouldn't help people. If we were doing this for real, we'd probably need a choose-your-own-adventure page "what are you testing?" "What operating system are you testing on?" That kind of thing. Have a "just show me the robots" escape hatch, and let people pick the one they want from a table of key packages. Do the classic up-sell and point them towards RobotLab because it has more check marks :) I definitely want to belay the Hub one, for now. I've made a totally self-contained jupyterhub work with ansible/systems spawner (haven't tried a k8s version yet). in certain kinds of training (schools), the participants may only have an older Chromebook, and wouldn't be able to install random software. Also building for ARM is still a touch outside of our capabilities. Anyhow: set up a NUC, run the installer, throw in your course content, turn on WiFi, and you're good to go in seconds per participant rather than tens-of-minutes.

…

On Thu, Oct 3, 2019, 00:22 Asko Soukka ***@***.***> wrote: Am I correct that ExoRobot does not yet include Selenium testing dependencies? I recall robot users have a long hoped for an installer with Selenium dependencies. But adding Firefox with geckodriver into mix would probably make ExoRobot fat? So, should there be "SeleniumRobot"? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#42?email_source=notifications&email_token=AAALCRCIQ3CBAGLUSQZUKOTQMVXQZA5CNFSM4IYDQJ6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAG5ZWI#issuecomment-537779417>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAALCRC53EF77KHDIGYLF4TQMVXQZANCNFSM4IYDQJ6A> .

[datakurre]

Ok Let's not guess too far ahead :) I could see two separate installer tracks: one for RobotLab and the other one for more conservative Robot Framework users. Quoting Nicholas Bollweg (2019-09-18 23:42:00)

MicrobotLab ð¬ * for CI environments

But "how micro" this could be, because "Lab"-branding would probably make this need nbconvert and nbrobot (the latter from robotkernel) to be able to execute notebooks. (I do actually use nbrobot in production RPA already.) No wx required here.

RobotLabLite ð

I'm not quite convinced that this is really needed. This could be used as a base to build custom environment, but why not just install miniconda then?

RobotLab ð * as it is now! + even shipping Firefox is great!

I agree everything on this. Big is good here. It's so good that the download is worth waiting for.

RobotHub ð¡ * ship a whole [3]littlest jupyter hub

This is interesting. I have never set up JupyterHub myself yet, but this would have very interesting educational and enterprise use cases :) Then the more conservative Robot Framework track. Robot Framework is completely missing standalone installers and your work seems to be the best there is so far. It feels a bit weird to design and build it here, but I understood that it would be better than forking this into another repository to follow. MiniRobot as it is. It's good to have something minimal with at least pip to install some libraries when you know what to do. ExoRobot was interesting. Like "MiniRobot" was just a proof-of-concept and "ExoRobot" was the actually practical base to build an environment. I feel this makes more sense thatn "RobotLabLite", but I'm not sure if ride really belongs to here (even when it comes with wx). WebRobot (terrible name). Comparable to RobotLab. Should come at least with Ride, SeleniumLibrary, geckodriver and Firefox. But maybe also with SSHLibrary, RESTinstance and SeleniumTestability. Really hard to draw line where to end here... Should I ask the other workshops at the next Robocon if they would like to have your installer experience?

[bollwyvl]

A lot to chew on there! I can rework some of the existing ones with your guidance above.... Adding the web one is a no-brainer, we already have everything. Maybe we need a further piece of the name on the non-lab ones, e.g WebRobotBox or something, as "*robot" is super overloaded, and we don't need to get into any trouble. The only bounding condition on us _shipping_ more installers is _supporting_ them, to my mind. What we would not want is upstreams getting complaints from users when things don't work, if the installers are not a suggested install mechanism. The workshop case is _ideal_, though, as the instructor is on the hook to provide support, and can't disappear. You should certainly offer it.

…

On Thu, Oct 3, 2019, 13:36 Asko Soukka ***@***.***> wrote: Ok Let's not guess too far ahead :) I could see two separate installer tracks: one for RobotLab and the other one for more conservative Robot Framework users. Quoting Nicholas Bollweg (2019-09-18 23:42:00) > MicrobotLab ð¬ > * for CI environments But "how micro" this could be, because "Lab"-branding would probably make this need nbconvert and nbrobot (the latter from robotkernel) to be able to execute notebooks. (I do actually use nbrobot in production RPA already.) No wx required here. > RobotLabLite ð I'm not quite convinced that this is really needed. This could be used as a base to build custom environment, but why not just install miniconda then? > RobotLab ð > * as it is now! > + even shipping Firefox is great! I agree everything on this. Big is good here. It's so good that the download is worth waiting for. > RobotHub ð¡ > * ship a whole [3]littlest jupyter hub This is interesting. I have never set up JupyterHub myself yet, but this would have very interesting educational and enterprise use cases :) Then the more conservative Robot Framework track. Robot Framework is completely missing standalone installers and your work seems to be the best there is so far. It feels a bit weird to design and build it here, but I understood that it would be better than forking this into another repository to follow. MiniRobot as it is. It's good to have something minimal with at least pip to install some libraries when you know what to do. ExoRobot was interesting. Like "MiniRobot" was just a proof-of-concept and "ExoRobot" was the actually practical base to build an environment. I feel this makes more sense thatn "RobotLabLite", but I'm not sure if ride really belongs to here (even when it comes with wx). WebRobot (terrible name). Comparable to RobotLab. Should come at least with Ride, SeleniumLibrary, geckodriver and Firefox. But maybe also with SSHLibrary, RESTinstance and SeleniumTestability. Really hard to draw line where to end here... Should I ask the other workshops at the next Robocon if they would like to have your installer experience? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#42?email_source=notifications&email_token=AAALCRDICGDT7ZE7NF53HK3QMYUSRA5CNFSM4IYDQJ6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAI7MKI#issuecomment-538048041>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAALCRDJHTFILBLS42GYN33QMYUSRANCNFSM4IYDQJ6A> .

[datakurre]

@bollwyvl To avoid misunderstandings, don’t waste your time to rework anything yet. As you said, all the pieces are already there and probably even I could put them together if I really need to. Let’s continue to focus on what this should mean for the “Lab” and only return to the generic installers when there is real need (and a possibly a maintainer).

Disclaimer about the CI-version: I don’t have personal use for it, because we have Nix on our CI machines :/