Skip to content

llm request from scratch <3 #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jsle97 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

llm request from scratch <3 #2

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions .askllm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# command: askllm

export ASKLLM_MISTRAL_API_KEY="mistral-api-key"
Copy link

Copilot AI Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API key is hardcoded as a placeholder value. This should be documented as needing user configuration, or the export should be removed entirely to require users to set their own key securely.

Suggested change
export ASKLLM_MISTRAL_API_KEY="mistral-api-key"
# NOTE: You must set ASKLLM_MISTRAL_API_KEY in your environment before using askllm.

Copilot uses AI. Check for mistakes.
export ASKLLM_MODEL="codestral-latest"
export ASKLLM_SYSTEM_PROMPT="You are an assistant created to help manage the Linux system in the terminal. Your task is to provide short and appropriate solutions to the problems described in the user's message without unnecessary testing. If you do not understand the task or have insufficient information, ask the user to provide more. Reply briefly and accurately, without using markdown or other text formatting - only plaintext."

askllm() {
local -a ROLES=()
local -a CONTENTS=()
ROLES+=("system")
CONTENTS+=("$ASKLLM_SYSTEM_PROMPT")

while true; do
if [[ -z "$ASKLLM_MISTRAL_API_KEY" ]]; then
echo -e "\033[31mERROR: ASKLLM_MISTRAL_API_KEY not set.\033[0m"
return 1
fi

echo -e "\033[36mEnter query (empty to exit, 'clear' to reset history):\033[0m"
USER_INPUT=$(cat)
Copy link

Copilot AI Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using cat without any input source will hang indefinitely waiting for stdin. This should be read -r USER_INPUT or read -p 'prompt: ' USER_INPUT to properly capture user input.

Suggested change
USER_INPUT=$(cat)
read -r USER_INPUT

Copilot uses AI. Check for mistakes.

if [[ -z "$USER_INPUT" ]]; then
echo -e "\033[33mExiting...\033[0m"
return 0
fi

if [[ "$USER_INPUT" == "clear" ]]; then
ROLES=("system")
CONTENTS=("$ASKLLM_SYSTEM_PROMPT")
echo -e "\033[33mHistory cleared.\033[0m"
continue
fi

echo -e "\n\033[33m---\033[0m"


ROLES+=("user")
CONTENTS+=("$USER_INPUT")

if ((${#ROLES[@]} > 7)); then
ROLES=("system" "${ROLES[@]: -6}")
CONTENTS=("${CONTENTS[0]}" "${CONTENTS[@]: -6}")
Comment on lines +40 to +42
Copy link

Copilot AI Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The history truncation logic is incorrect. When truncating to keep the last 6 exchanges plus the system prompt, the CONTENTS array should preserve the system prompt at index 0, but line 42 assumes CONTENTS[0] is always the system content which may not be true after multiple truncations.

Suggested change
if ((${#ROLES[@]} > 7)); then
ROLES=("system" "${ROLES[@]: -6}")
CONTENTS=("${CONTENTS[0]}" "${CONTENTS[@]: -6}")
CONTENTS=("$ASKLLM_SYSTEM_PROMPT" "${CONTENTS[@]: -6}")

Copilot uses AI. Check for mistakes.
fi

local MESSAGES="[]"
for i in "${!ROLES[@]}"; do
MESSAGES=$(jq --arg role "${ROLES[$i]}" --arg content "${CONTENTS[$i]}" \
'. += [{"role": $role, "content": $content}]' <<< "$MESSAGES")
done

RESPONSE=$(curl -sS --connect-timeout 10 --max-time 30 \
"https://api.mistral.ai/v1/chat/completions" \
-H "Authorization: Bearer $ASKLLM_MISTRAL_API_KEY" \
-H "Content-Type: application/json" \
-d "$(jq -n \
--argjson messages "$MESSAGES" \
--arg model "$ASKLLM_MODEL" \
'{
model: $model,
messages: $messages,
temperature: 0.3,
max_tokens: 512,
stream: false
}')")

Copy link

Copilot AI Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error handling doesn't distinguish between API errors and missing response content. Consider checking for API error responses first (e.g., .error.message) before attempting to extract the content.

Suggested change
ERROR_MSG=$(echo "$RESPONSE" | jq -r '.error.message // empty')
if [[ -n "$ERROR_MSG" ]]; then
echo -e "\033[31mAPI Error: $ERROR_MSG\033[0m"
echo -e "\033[33mAPI response:\n$RESPONSE\033[0m"
return 1
fi

Copilot uses AI. Check for mistakes.
ANSWER=$(echo "$RESPONSE" | jq -r '.choices[0].message.content // empty')
if [[ -z "$ANSWER" ]]; then
echo -e "\033[31mError: No LLM response.\033[0m"
echo -e "\033[33mAPI response:\n$RESPONSE\033[0m"
return 1
fi

ROLES+=("assistant")
CONTENTS+=("$ANSWER")

echo -e "\n\033[32m--- Response ---\033[0m"
echo "$ANSWER" | fold -s -w "$(tput cols)" | while IFS= read -r line; do
echo -e "\033[32m$line\033[0m"
done
done
}