Project
LexMemoryBench: Household Long-Term Memory
A synthetic-data pipeline for evaluating cross-member, cross-session memory in household robots and assistants.
Python · Typed Schemas · Prompt Engineering · Synthetic Data QA
Product Snapshot
- Role
- Evaluation pipeline engineering / team project
- Users
- Teams researching long-term memory, household robots, and multi-user agents.
- Stage
- The core generation pipeline runs; the complete benchmark haystack and scaled evaluation remain in progress.
- Focus
- A traceable chain across personas, QA, evidence, evidence sessions, and filler sessions.
- Validation
- Implemented a five-stage typed pipeline that keeps household members, memory facts, and conversational evidence distinguishable and auditable.
- Public Proof
- 15 attributable repository commits, pipeline documentation, and staged artifacts; the current Step-3 bundle is not a final benchmark result.
Next Step
Beyond remembering one user
A household robot may speak separately with several family members. A fact such as “I dislike cilantro” is useful only when the system preserves who said it, when it was said, and what evidence supports it.
LexMemoryBench focuses on cross-member, cross-session memory. Retrieval is not enough: attribution and evidence must also remain correct.
The pipeline I helped build
The current generation chain has five traceable stages:
- household and member personas
- constrained questions and answers
- minimal supporting evidence
- natural evidence-bearing conversations
- filler conversations that do not leak answers
Typed schemas, raw model envelopes, and quality checks are retained between stages so failures can be traced to persona, evidence, or dialogue generation.
My role
This is a team project. My work centers on engineering the synthetic-data pipeline, schema constraints, prompt discipline, and documentation. The repository shows 15 attributable commits. I do not claim the complete benchmark or other researchers’ work as an individual result.
Current boundary
The core pipeline runs and has produced staged submission artifacts. A complete LongMemEval-style haystack, scaled runs, and final comparisons are not finished, so this page presents verifiable system assets rather than a model ranking.
Why it matters
For multi-user memory, the most harmful error is often not forgetting. It is giving one person’s information to another. The project connects privacy, evidence attribution, and evaluation design in one pipeline.
Configure the public Giscus environment variables to open discussion on the public site.