Quote:
Originally Posted by Ekco
[You must be logged in to view images. Log in or Register.]
and in theory actually index and parse the documents i have in the data folder locally so i can ask it questions
|
this is the part i'm currently stuck on and have taken a break, they just released a new Nvida driver and python version and yada yada so i had to go through and update all that and make sure it still works, then the actual QA/bug fixing of checking if the model is getting the information it's spitting back from it's pre-trained model, the actual pdfs in it's folder or just straight up hallucinating
which has been a major headache so far
i have an entire document of testing i run it through
Quote:
### Kaia Functionality Testing Scenarios
This document outlines a series of tests to verify Kaia's core capabilities, including Retrieval-Augmented Generation (RAG), command execution, memory management, and persona adherence.
1. RAG (Retrieval-Augmented Generation) Tests
These tests verify Kaia's ability to retrieve and synthesize information from its knowledge base (./data and ./personal_context directories). Ensure the relevant files are present in these directories and the index has been rebuilt.
From 'Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig.pdf' (or its .txt equivalent):
Query: What are the four approaches to AI as described by Russell and Norvig?
Expected: Kaia lists the four approaches (acting humanly, thinking humanly, acting rationally, thinking rationally) and provides brief explanations for each, without self-referential statements about being an AI.
|
the persona adherence section is fun,
[You must be logged in to view images. Log in or Register.]