Skip to content

Commit da99c8b

Browse files
committed
SimpleChatTC: Update readme a bit
1 parent 05b52c3 commit da99c8b

File tree

1 file changed

+25
-14
lines changed

1 file changed

+25
-14
lines changed

tools/server/public_simplechat/readme.md

Lines changed: 25 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -42,8 +42,9 @@ any adaptive culling of old messages nor of replacing them with summary of their
4242
is a optional sliding window based chat logic, which provides a simple minded culling of old messages from
4343
the chat history before sending to the ai model.
4444

45-
NOTE: Wrt options sent with the request, it mainly sets temperature, max_tokens and optionaly stream for now.
46-
However if someone wants they can update the js file or equivalent member in gMe as needed.
45+
NOTE: Wrt options sent with the request, it mainly sets temperature, max_tokens and optionaly stream as well
46+
as tool_calls mainly for now. However if someone wants they can update the js file or equivalent member in
47+
gMe as needed.
4748

4849
NOTE: One may be able to use this to chat with openai api web-service /chat/completions endpoint, in a very
4950
limited / minimal way. One will need to set model, openai url and authorization bearer key in settings ui.
@@ -55,7 +56,7 @@ One could run this web frontend directly using server itself or if anyone is thi
5556
frontend to configure the server over http(s) or so, then run this web frontend using something like python's
5657
http module.
5758

58-
### running using tools/server
59+
### running directly using tools/server
5960

6061
./llama-server -m path/model.gguf --path tools/server/public_simplechat [--port PORT]
6162

@@ -78,10 +79,15 @@ remember to
7879

7980
* use a GenAi/LLM model which supports tool calling.
8081

81-
* if fetch web url / page tool call is needed, remember to run the bundled local.tools/simpleproxy.py helper
82+
* if fetch web url / page tool call is needed remember to run the bundled local.tools/simpleproxy.py
83+
helper along with its config file
8284

83-
* remember that this is a relatively dumb proxy logic along with optional stripping of scripts/styles/headers/footers/...,
84-
Be careful if trying to fetch web pages, and use it only with known safe sites.
85+
* cd tools/server/public_simplechat/local.tools; python3 ./simpleproxy.py --config simpleproxy.json
86+
87+
* remember that this is a relatively dumb proxy logic along with optional stripping of scripts / styles
88+
/ headers / footers /..., Be careful if trying to fetch web pages, and use it only with known safe sites.
89+
90+
* it allows one to specify a white list of allowed.domains, look into local.tools/simpleproxy.json
8591

8692
### using the front end
8793

@@ -312,7 +318,9 @@ wrt repeatations in general in the generated text response.
312318
A end-user can change these behaviour by editing gMe from browser's devel-tool/console or by
313319
using the provided settings ui (for settings exposed through the ui). The logic uses a generic
314320
helper which autocreates property edit ui elements for the specified set of properties. If the
315-
new property is a number or text or boolean, the autocreate logic will handle it.
321+
new property is a number or text or boolean or a object with properties within it, autocreate
322+
logic will try handle it automatically. A developer can trap this autocreation flow and change
323+
things if needed.
316324

317325

318326
### OpenAi / Equivalent API WebService
@@ -362,22 +370,22 @@ server logic, this helps bypass the CORS restrictions applied if trying to direc
362370
browser js runtime environment. Depending on the path specified wrt the proxy server, if urltext
363371
(and not urlraw), it additionally tries to convert html content into equivalent text to some extent
364372
in a simple minded manner by dropping head block as well as all scripts/styles/footers/headers/nav.
365-
May add support for white list of allowed sites to access or so.
366373
* the logic does a simple check to see if the bundled simpleproxy is running at specified fetchProxyUrl
367374
before enabling fetch web related tool calls.
368375
* The bundled simple proxy can be found at
369376
* tools/server/public_simplechat/local.tools/simpleproxy.py
377+
* it provides for a basic white list of allowed domains to access or so
370378

371379

372380
#### Extending with new tools
373381

374382
Provide a descriptive meta data explaining the tool / function being provided for tool calling,
375383
as well as its arguments.
376384

377-
Provide a handler which should implement the specified tool / function call or rather constructs
378-
the code to be run to get the tool / function call job done, and inturn pass the same to the
379-
provided web worker to get it executed. Remember to use console.log while generating any response
380-
that should be sent back to the ai model, in your constructed code.
385+
Provide a handler which should implement the specified tool / function call or rather for many
386+
cases constructs the code to be run to get the tool / function call job done, and inturn pass
387+
the same to the provided web worker to get it executed. Remember to use console.log while
388+
generating any response that should be sent back to the ai model, in your constructed code.
381389

382390
Update the tc_switch to include a object entry for the tool, which inturn includes
383391
* the meta data as well as
@@ -415,14 +423,17 @@ so that any scheduled asynchronous code or related async error handling using pr
415423
gets executed, before tool calling returns and thus data / error generated by those async code also
416424
get incorporated in result sent to ai engine on the server side.
417425

418-
#### ToDo
419426

420-
Is the promise land trap deep enough, need to think through and explore around this once later.
427+
### ToDo
428+
429+
Is the tool call promise land trap deep enough, need to think through and explore around this once later.
421430

422431
Trap error responses.
423432

424433
Handle reasoning/thinking responses from ai models.
425434

435+
Handle multimodal handshaking with ai models.
436+
426437

427438
### Debuging the handshake
428439

0 commit comments

Comments
 (0)