@@ -42,8 +42,9 @@ any adaptive culling of old messages nor of replacing them with summary of their
4242is a optional sliding window based chat logic, which provides a simple minded culling of old messages from
4343the chat history before sending to the ai model.
4444
45- NOTE: Wrt options sent with the request, it mainly sets temperature, max_tokens and optionaly stream for now.
46- However if someone wants they can update the js file or equivalent member in gMe as needed.
45+ NOTE: Wrt options sent with the request, it mainly sets temperature, max_tokens and optionaly stream as well
46+ as tool_calls mainly for now. However if someone wants they can update the js file or equivalent member in
47+ gMe as needed.
4748
4849NOTE: One may be able to use this to chat with openai api web-service /chat/completions endpoint, in a very
4950limited / minimal way. One will need to set model, openai url and authorization bearer key in settings ui.
@@ -55,7 +56,7 @@ One could run this web frontend directly using server itself or if anyone is thi
5556frontend to configure the server over http(s) or so, then run this web frontend using something like python's
5657http module.
5758
58- ### running using tools/server
59+ ### running directly using tools/server
5960
6061./llama-server -m path/model.gguf --path tools/server/public_simplechat [ --port PORT]
6162
@@ -78,10 +79,15 @@ remember to
7879
7980* use a GenAi/LLM model which supports tool calling.
8081
81- * if fetch web url / page tool call is needed, remember to run the bundled local.tools/simpleproxy.py helper
82+ * if fetch web url / page tool call is needed remember to run the bundled local.tools/simpleproxy.py
83+ helper along with its config file
8284
83- * remember that this is a relatively dumb proxy logic along with optional stripping of scripts/styles/headers/footers/...,
84- Be careful if trying to fetch web pages, and use it only with known safe sites.
85+ * cd tools/server/public_simplechat/local.tools; python3 ./simpleproxy.py --config simpleproxy.json
86+
87+ * remember that this is a relatively dumb proxy logic along with optional stripping of scripts / styles
88+ / headers / footers /..., Be careful if trying to fetch web pages, and use it only with known safe sites.
89+
90+ * it allows one to specify a white list of allowed.domains, look into local.tools/simpleproxy.json
8591
8692### using the front end
8793
@@ -312,7 +318,9 @@ wrt repeatations in general in the generated text response.
312318A end-user can change these behaviour by editing gMe from browser's devel-tool/console or by
313319using the provided settings ui (for settings exposed through the ui). The logic uses a generic
314320helper which autocreates property edit ui elements for the specified set of properties. If the
315- new property is a number or text or boolean, the autocreate logic will handle it.
321+ new property is a number or text or boolean or a object with properties within it, autocreate
322+ logic will try handle it automatically. A developer can trap this autocreation flow and change
323+ things if needed.
316324
317325
318326### OpenAi / Equivalent API WebService
@@ -362,22 +370,22 @@ server logic, this helps bypass the CORS restrictions applied if trying to direc
362370browser js runtime environment. Depending on the path specified wrt the proxy server, if urltext
363371(and not urlraw), it additionally tries to convert html content into equivalent text to some extent
364372in a simple minded manner by dropping head block as well as all scripts/styles/footers/headers/nav.
365- May add support for white list of allowed sites to access or so.
366373* the logic does a simple check to see if the bundled simpleproxy is running at specified fetchProxyUrl
367374 before enabling fetch web related tool calls.
368375* The bundled simple proxy can be found at
369376 * tools/server/public_simplechat/local.tools/simpleproxy.py
377+ * it provides for a basic white list of allowed domains to access or so
370378
371379
372380#### Extending with new tools
373381
374382Provide a descriptive meta data explaining the tool / function being provided for tool calling,
375383as well as its arguments.
376384
377- Provide a handler which should implement the specified tool / function call or rather constructs
378- the code to be run to get the tool / function call job done, and inturn pass the same to the
379- provided web worker to get it executed. Remember to use console.log while generating any response
380- that should be sent back to the ai model, in your constructed code.
385+ Provide a handler which should implement the specified tool / function call or rather for many
386+ cases constructs the code to be run to get the tool / function call job done, and inturn pass
387+ the same to the provided web worker to get it executed. Remember to use console.log while
388+ generating any response that should be sent back to the ai model, in your constructed code.
381389
382390Update the tc_switch to include a object entry for the tool, which inturn includes
383391* the meta data as well as
@@ -415,14 +423,17 @@ so that any scheduled asynchronous code or related async error handling using pr
415423gets executed, before tool calling returns and thus data / error generated by those async code also
416424get incorporated in result sent to ai engine on the server side.
417425
418- #### ToDo
419426
420- Is the promise land trap deep enough, need to think through and explore around this once later.
427+ ### ToDo
428+
429+ Is the tool call promise land trap deep enough, need to think through and explore around this once later.
421430
422431Trap error responses.
423432
424433Handle reasoning/thinking responses from ai models.
425434
435+ Handle multimodal handshaking with ai models.
436+
426437
427438### Debuging the handshake
428439
0 commit comments