Skip to content
aroa.ai
ES

model card

Aroa Index

This is what in a normal machine learning repo would be the model's README: what it is, what it was trained on, what decisions were made and where you shouldn't trust it. Here it's also, with no metaphor, my “about me”.

There are four entries because four things were tried. Only one is in production; the others stayed documented instead of being deleted, which is where you actually learn something.

Each model is named after its stage in a search engine: index builds the inverted index, vector tries embeddings, rank scores with BM25, infer generates in your browser and serve would generate from a server. Lined up they are the whole pipeline, and the name of the one in production — index — says what is really inside: not a brain, a well-built data structure.

model what it is size acc coverage rejection
Aroa Index in production Retrieval (TF-IDF) 512 KB 90% 90% 76%
Aroa Infer optional The above + an LLM that rewrites ~2 GB — — —
Aroa Vector discarded Embeddings (e5-small) 1,1 MB 88% 82% 76%
Aroa Rank discarded BM25 561 KB 86% 92% 76%

All three figures come from the same evaluation set. Coverage is how many legitimate questions it answers instead of saying “I don't know”; rejection, how many out-of-domain questions it correctly turns down. Full detail at /eval.

Aroa Index Pure retrieval. This site's actual model. in production

index for the inverted index: what training produces is literally a map from terms to documents, with weights. It does not reason, it looks things up — and the name says so, to avoid promising more.

version
1.7.2
architecture
TF-IDF (palabras 1-2 + caracteres 3-5) + similitud del coseno
documents
112
vocabulary
18125 terms
trained
2026-08-17
threshold
0.1812

What it does

It's an information retrieval model, not a language model. It turns each question into a TF-IDF vector (1-2 gram words + 3-5 character n-grams, to survive missing accents and typos), compares it by cosine similarity against the corpus, and returns the closest document. It doesn't generate text: it can't say anything that isn't already written.

Aroa Xinping trained it with scikit-learn. Inference runs entirely in your browser, with no server and no API: the artefact downloads as a JSON file and from then on nothing leaves your device.

Training data

The corpus is one real person's path, not a generic dataset: science baccalaureate, then arts, three years of Fine Arts, two of a vocational degree in Dietetics, and now the Data Science degree at the UOC alongside the Ironhack bootcamp. Each stage was left for a specific reason, and those reasons are in the corpus exactly as someone would ask about them.

On top of that: her own projects, technical opinions, how her content works on social media, and metadata about the model itself. No document comes from scraping — they were all written by the person they describe, and she has reviewed them.

Design decisions

TF-IDF and not an LLM. A large language model behind this domain would have been someone else's API with her name on it. TF-IDF is simpler, but every piece — vectoriser, weights, threshold — is hers, trained on her own data.

Inference in the browser. With no backend there is no cost per visit, no network latency, and nobody's data leaving their device. The price: the model has to fit in a reasonable JSON file, so the vectors are pruned and rounded.

Calibrated threshold, not eyeballed. Training evaluates against paraphrased questions and against out-of-domain questions, and picks the cut that best balances both errors. It would rather stay quiet than make things up.

Interpretable by construction, not by tooling. Working out what a large language model is “thinking” at an intermediate layer takes purpose-built techniques like the logit lens, which translate its internal vectors into words. They exist because a neural network is opaque inside. Here they are not needed: every answer arrives with its five candidate documents, their scores and the exact terms that tipped the balance. Interpretability is not a layer bolted on top — it is a property of having picked a model you can read.

Capabilities

  • Answers about background, studies, projects and technical opinions in the corpus.
  • Handles questions without accents, with typos, or phrased differently from the original document.
  • Shows its confidence score on every answer: no black box.
  • Works offline once downloaded; depends on no API.
  • Accepts spoken questions where the browser supports it.

Limitations

  • It has no memory between questions: every query is resolved from scratch.
  • It only knows what is in its corpus — dozens of documents about one person, not a general knowledge base.
  • It doesn't generate text: if the question matches nothing, it says so.
  • It's biased towards the opinions of the person who trained it, because she wrote them. It isn't neutral and doesn't pretend to be.
  • The facts are Aroa's and she has reviewed them, but the wording is from whoever assembled the corpus.

History

  1. v1.7.2 2026-08-17 acc 90% · cob 90% · rech 76% · 112 docs
  2. v1.7.1 2026-08-17 acc 90% · cob 90% · rech 76% · 112 docs

    Arreglados los 12 fallos del eval cubriendo los huecos de vocabulario que los causaban, más el colapso de letras repetidas en el analizador para que «holaaaa» encuentre el saludo. Pero el 100% que salió primero era falso: las variantes añadidas eran las preguntas del eval casi literales, o sea enseñarle el examen. Al auditarlo aparecieron 23 preguntas del eval español y 15 del inglés copiadas del corpus, algunas de antes — mientras /eval afirmaba que ninguna lo estaba. Reescritas todas como paráfrasis de verdad, y con scripts/fuga_eval.py para que no vuelva a pasar. Las cifras honestas: 88.5% → 90.4% en español y 88.1% → 94.0% en inglés, medidas contra un eval que ya no está contaminado.

  3. v1.7.0 2026-08-17 acc 100% · cob 98% · rech 76% · 112 docs
  4. v1.6.0 2026-08-17 acc 88% · cob 91% · rech 76% · 112 docs

    112 documentos, con trece conceptos técnicos nuevos —regresión, estadística, visualización, git, la nube, notebooks, correlación— que es el pilar de contenido que ya hace en vídeo. Y seis fusiones más: el script scripts/solapes.py, escrito después de meter duplicados a mano dos tandas seguidas, encontró cuatro pares que se me habían pasado y una contradicción de verdad — un documento afirmaba que la estadística es la rama que mejor se le da mientras otro contaba que la inferencial la puso en su sitio. Buscar antes de escribir hace que añadir documentos SUBA la accuracy: 86.5% → 88.5% en español, 86.9% → 88.1% en inglés.

  5. v1.5.1 2026-08-17 acc 87% · cob 91% · rech 76% · 105 docs

    Cuatro duplicados fuera. Al ampliar el corpus se habían escrito documentos que ya existían con otro id —«de dónde eres» estaba dos veces, y «dónde vives» también— y eso es justo lo que degrada un modelo de recuperación: dos documentos compitiendo por las mismas palabras. Fusionados, la accuracy sube de 85.6% a 86.5% en español y de 84.5% a 86.9% en inglés, con la cobertura intacta. La lección: antes de añadir, buscar si ya está.

  6. v1.5.0 2026-08-17 acc 86% · cob 71% · rech 96% · 109 docs

    Corpus a 109 documentos y dos cambios de fondo. PESO_CHAR sube de 0.3 a 0.4 para aguantar faltas de ortografía: elegido con una rejilla medida contra dos sets a la vez, el normal y una copia con erratas metidas a propósito. Y la calibración del umbral se reescribió — maximizaba la media de aceptadas y rechazadas, y a esta escala dos cortes separados por una milésima daban 71% y 91% de cobertura. Ahora maximiza cobertura con un suelo duro de rechazo del 75%, que no se puede compensar.

  7. v1.4.0 2026-08-17 acc 88% · cob 93% · rech 76% · 96 docs
  8. v1.3.0 2026-08-17 acc 88% · cob 90% · rech 80% · 96 docs

    Corpus de 69 a 96 documentos, con el hueco de «técnico» (6 → 16) y un tema nuevo, «opiniones», que no existía. Medido contra el eval anterior para que la comparación sea justa, el coste real es 2,3 puntos de accuracy y 4 de rechazo — a cambio de 27 temas que antes no tenían respuesta. La caída aparente de 7 puntos era un artefacto de haber ampliado el eval a la vez.

  9. v1.2.2 2026-08-17 acc 95% · cob 94% · rech 84% · 69 docs
  10. v1.2.0 2026-08-17 acc 95% · cob 94% · rech 84% · 69 docs

    Corpus de 63 a 69 documentos: seis proyectos que existían en GitHub pero de los que el modelo no sabía nada. El documento de proyectos decía «ahora mismo tres» cuando la web ya enseñaba doce — el modelo contradecía a su propia web. Las descripciones salen de los README de cada repo, no de memoria. Accuracy 94.9% → 95.4%, pero el rechazo fuera de dominio baja de 88% a 84%: con más documentos, el umbral que mejor equilibra los dos errores cae, y el modelo se calla algo menos.

  11. v1.1.1 2026-08-17 acc 95% · cob 94% · rech 88% · 63 docs
  12. v1.1.0 2026-08-16 acc 95% · cob 94% · rech 88% · 63 docs

    48 question variants added to the 16 documents that were still failing. Coverage 90% to 94%. Again: no algorithm change.

  13. v1.0.0 2026-08-16 acc 97% · cob 90% · rech 88% · 63 docs

    Corpus expanded from 50 to 63 documents with data Aroa provided, and two published false facts corrected (she lives in Valencia, not Barcelona; and she does not listen to the music /about claimed). Five vocabulary collisions fixed. Coverage rises from 73% to 90% without touching the algorithm: the biggest jump in the whole project came from working the data.

  14. v0.9.1 2026-08-13 acc 98% · cob 73% · rech 97% · 50 docs

    22 real conversational questions added to the out-of-domain set. Coverage drops from 91% to 73% — the model did not get worse, the measurement got honest.

  15. v0.9.0 2026-08-13 acc 98% · cob 91% · rech 93% · 50 docs

    First version. The out-of-domain evaluation was eight made-up questions, which was far too optimistic: nobody writes like that.

Aroa Infer Same thing, but a local LLM writes the answer. optional

infer for inference: it is the only variant that runs a language model on your machine instead of querying a pre-computed index. Hence the two gigabytes and the two minutes.

model
Qwen2.5-3B-Instruct
runs on
your browser (WebGPU)
download
~2 GB
first answer
> 2 min
subsequent
10-20 s

Retrieval still picks the documents exactly as before; the LLM only rewrites them. It's RAG, not free conversation: it contributes no data, it only rephrases what was already there. If retrieval doesn't clear the threshold, this variant generates nothing either — you get the same “I don't know”, because handing an LLM empty context to improvise on is precisely what this site promises not to do.

Why Qwen and not Llama

The first version used Llama-3.2-1B. Measured in the browser with the real corpus in front of it, it refused to answer a plain “hello” (“I cannot comply with that request”), contradicted the very document it was given (“I am not Aroa Xinping, I am a language model”) and mixed documents up: asked why she left Fine Arts, it answered “I felt more comfortable at the gym”, in the masculine. Qwen2.5-3B answers the same questions faithfully.

An honest warning

The model that answers well is too slow and the one that answers fast doesn't answer well. This variant shows something real — an LLM running entirely in the browser, with no server — but it is not a good user experience, and isn't sold as one. That's why you have to turn it on by hand and why the default mode is the other one.

Aroa Vector Embeddings instead of TF-IDF. Worse at rejecting. descartado

vector for the vector store: it is the dense retrieval stage, the one comparing meanings instead of words. It understood paraphrase better, and for that very reason it could not tell when to stay quiet.

architecture
embeddings (intfloat/multilingual-e5-small) + similitud del coseno
accuracy@1
88%
coverage
82%
rejection
76%

Embeddings understand paraphrase better, but keeping quiet costs them far more: sentence cosine with these models compresses into a narrow band whether the question fits or not — anisotropy. With the threshold calibrated, both reject the same share of out-of-domain questions, and that is the telling part: to get there, the embedding model has to set the cut so high that it also throws away legitimate questions, and it answers noticeably fewer than TF-IDF. The cost does not show up in rejection, it shows up in coverage. TF-IDF, comparing literal words, gives genuine zero similarity between sentences with nothing in common, and that clean separation is what lets it stay quiet with confidence.

Artefact published at /model/aroa-vector.json.

History

  1. v1.2.0 2026-08-17 acc 88% · cob 82% · rech 76% · 112 docs
  2. v1.1.0 2026-08-17 acc 87% · cob 59% · rech 100% · 96 docs

    Reentrenado a 96 documentos para comprobar si a esta escala los embeddings adelantaban a TF-IDF. No: la cobertura cae al 58,7% — pero medido con el set de evaluación anterior, el que todavía tenía preguntas copiadas del corpus, así que no se puede comparar con lo de después. Medido de nuevo con el examen actual, de 96 a 112 documentos la cobertura de los embeddings pasa de 80,8% a 81,7%: una pregunta, ruido. La anisotropía no se agrava con la escala en este rango, se mantiene — y con ella los nueve puntos de cobertura que le saca TF-IDF en las dos tallas.

  3. v1.0.1 2026-08-16 acc 97% · cob 63% · rech 100% · 63 docs

    Retrained on the updated corpus so the comparison against aroa index runs on the same corpus and the same eval. Accuracy goes up but coverage stays at 63%: the embeddings' problem was never understanding, it was knowing when to stay quiet.

  4. v1.0.0 2026-08-14 acc 95% · cob 73% · rech 84% · 63 docs

    Embeddings instead of TF-IDF. Better at paraphrase, keeping quiet costs them far more. Discarded.

Aroa Rank BM25, the industry standard. A technical draw. descartado

rank for ranking function, the formal category BM25 belongs to. Its surname, basically.

architecture
BM25 (k1=1.2, b=0.0) sobre palabras 1-2 + caracteres 3-5
accuracy@1
86%
coverage
92%
rejection
76%

BM25 is what Elasticsearch, Lucene and Solr run inside: it saturates term repetition and models document length. The parameters were picked by grid search over this corpus rather than taken as defaults — and it came out at b=0, meaning length normalisation actually gets in the way here. That makes sense: a long document is long because it has content, not filler.

It answers more questions than TF-IDF — better coverage — but it picks the right document less often, at the same out-of-domain rejection: what it gains in quantity it loses in aim. Neither wins clearly and the difference fits inside the noise of the evaluation set, so rewriting the whole inference layer wasn't worth it. Artefact at /model/aroa-rank.json.

History

  1. v1.2.0 2026-08-17 acc 86% · cob 92% · rech 76% · 112 docs
  2. v1.1.0 2026-08-17 acc 86% · cob 82% · rech 96% · 96 docs

    Reentrenado a 96 documentos. Sigue perdiendo en «útil», pero su rechazo sube al 96% mientras el de TF-IDF baja al 80%: la brecha pasa de 4 puntos a 16. Es el dato que habrá que volver a mirar si el corpus sigue creciendo.

  3. v1.0.1 2026-08-16 acc 94% · cob 86% · rech 92% · 63 docs

    Retrained on the updated corpus for the same reason as aroa vector. Still better at rejecting what it does not know and worse at covering what it does: the same trade-off it already had.

  4. v1.0.0 2026-08-16 acc 95% · cob 95% · rech 80% · 63 docs

    BM25 with k1 and b picked by grid search. Technical tie with TF-IDF. Discarded, because switching meant rewriting the whole inference for nothing measurable.

The conclusion that matters most

Both algorithm changes moved within the noise. Growing the corpus did not: from 50 to 112 documents, everything measured on the same exam, the questions that end up with the right answer in front of them go from 35.9% to 81.7% — 45.7 points. Changing the algorithm moved under 5, and downwards. But the detail matters: on the questions that already had their document at 50, growing the corpus barely changed anything (+4.3 points, close to noise). The whole gain is in being able to answer 42 questions that previously had no document to point at. Growing the corpus does not improve the answers you already give: it lets you give the ones that were missing.

Intended use

Portfolio and point of contact: learn about her background, see her projects, consider a collaboration. For press, collaborations or hiring, the model itself can give you her contact — just ask it.

← back to talking with Aroa Index