The whole of this run is a release asset
This site holds 0 of the 9 question directories this run wrote; the archive holds every one:
- archive
- minidev-sqlite-mistralai-mixtral-8x7b-instru-4.tar.gz
- size
- 1,075,349 bytes
- sha256
- 7f34c38f4a2fbc7c9cfd7720ee1dcc8af8c27748e47e2bdd6ebe556f7e3703ea
What it counted
| what | how many |
|---|---|
| questions audited | 30 |
| NOT_EQUAL | 9 |
| ERROR | 18 |
| EQUAL | 3 |
| probes fired | 0 |
the probes
| probe | fired on |
|---|---|
| ordering-over-numeric-text | 0 |
| arbitrary-cut | 0 |
| not-a-function-of-the-data | 0 |
| float-aggregate-order | 0 |
| direction-against-question | 0 |
credited by BIRD's own check and NOT_EQUAL here
| what | how many |
|---|---|
| credited by BIRD and NOT_EQUAL here | 0 |
| multiplicity | 0 |
| type | 0 |
| order | 0 |
| truncation | 0 |
| other | 0 |
| the test-suite check answered 1 | 0 |
| the test-suite check answered 0 | 0 |
What the run was made of
- run
- audit-91b4d389-5897-4df0-8417-7c6f3e046757
- summary format
- attestql/audit/summary/2
- question file
- data/questions/mini_dev_sqlite.json sha256:4ba5fa8de55856222f484d380d2ba872b380bf79d825de70478e2120cb0fc43b | https://bird-bench.oss-cn-beijing.aliyuncs.com/minidev.zip (sha256 cc48ba16838204e4e214512030cb572eeb5f7bcdd999bae4b9b6ff12ec13b92f, downloaded 2026-09-07), member minidev/MINIDEV/mini_dev_sqlite.json, 2024-06-19
- questions in the file
- 500
- prediction file
- data/preds-sqlite/predict_mini_dev_mistralai-mixtral-8x7b-instru-4_sqlite.json sha256:53cdf9142e3f2e49941eb7b50ebbea90361b2c5b46de37d5ba1c244187d643a2 | https://raw.githubusercontent.com/bird-bench/mini_dev/b3d4bcbbae9a96934ad812551eb400c7a3b23c12/llm/exp_result/sql_output_kg/predict_mini_dev_mistralai-mixtral-8x7b-instru-4_sqlite.json, 2024-06-19
- statements read
- 500
- keyed by
- position
- data file
- data/minidev/dev_databases/financial/financial.sqlite sha256:d15d89cdb068a202b6f2b99342af44dffc1d52545b39ceaf62efdc0ba570101e | minidev.zip (sha256 cc48ba16838204e4e214512030cb572eeb5f7bcdd999bae4b9b6ff12ec13b92f), member minidev/MINIDEV/dev_databases, financial/financial.sqlite, 2024-06-13
- server
- SQLite 3.53.4 | file=/private/tmp/attestql-runs/data/minidev/dev_databases/financial/financial.sqlite | size=71294976
- database role
- file
- engine
- sqlite
- parser
- validator audit:sqlglot-sqlite-parse, sqlglot 30.18.0, dialect sqlite
- serialization
- attestql/audit/2
- statement timeout
- 30 s
- fixture digest depth
- counts
- schema digest
- sha256:dec3b7590cdaf8df74c4ddbef766cac24a33554a327998fe33ef165b15bdf90a
- data as of
- 2026-09-08T05:29:04.478800+00:00 the instant the run started
- shuffled copies
- prepared
- shuffle seed
- 1
- experimental probe
- off
The session the run was made in
what the engine reported, 9 settings
- sqlite_version
- 3.53.4
- encoding
- UTF-8
- reverse_unordered_selects
- 0
- query_only
- 1
- journal_mode
- delete
- data_version
- 1
- compile_options
- ATOMIC_INTRINSICS=1,COMPILER=clang-21.0.0,DEFAULT_AUTOVACUUM,DEFAULT_CACHE_SIZE=-2000,DEFAULT_FILE_FORMAT=4,DEFAULT_JOURNAL_SIZE_LIMIT=-1,DEFAULT_MMAP_SIZE=0,DEFAULT_PAGE_SIZE=4096,DEFAULT_PCACHE_INITSZ=20,DEFAULT_RECURSIVE_TRIGGERS,DEFAULT_SECTOR_SIZE=4096,DEFAULT_SYNCHRONOUS=2,DEFAULT_WAL_AUTOCHECKPOINT=1000,DEFAULT_WAL_SYNCHRONOUS=2,DEFAULT_WORKER_THREADS=0,DIRECT_OVERFLOW_READ,ENABLE_API_ARMOR,ENABLE_COLUMN_METADATA,ENABLE_DBSTAT_VTAB,ENABLE_FTS3,ENABLE_FTS3_PARENTHESIS,ENABLE_FTS5,ENABLE_GEOPOLY,ENABLE_MATH_FUNCTIONS,ENABLE_MEMORY_MANAGEMENT,ENABLE_PERCENTILE,ENABLE_PREUPDATE_HOOK,ENABLE_RTREE,ENABLE_SESSION,ENABLE_STAT4,ENABLE_UNLOCK_NOTIFY,MALLOC_SOFT_LIMIT=1024,MAX_ATTACHED=10,MAX_COLUMN=2000,MAX_COMPOUND_SELECT=500,MAX_DEFAULT_PAGE_SIZE=8192,MAX_EXPR_DEPTH=1000,MAX_FUNCTION_ARG=1000,MAX_LENGTH=1000000000,MAX_LIKE_PATTERN_LENGTH=50000,MAX_MMAP_SIZE=0x7fff0000,MAX_PAGE_COUNT=0xfffffffe,MAX_PAGE_SIZE=65536,MAX_SQL_LENGTH=1000000000,MAX_TRIGGER_DEPTH=1000,MAX_VARIABLE_NUMBER=250000,MAX_VDBE_OP=250000000,MAX_WORKER_THREADS=8,MUTEX_PTHREADS,SYSTEM_MALLOC,TEMP_STORE=1,THREADSAFE=1,USE_URI
- collation_list
- RTRIM,NOCASE,BINARY
- case_sensitive_like
- 0
What the run states about itself
- ids the question file states twice
- 137, 138
- prediction positions not compared
- 487, 488
The questions
| question | rule | verdict | class | probes | what was asked |
|---|---|---|---|---|---|
| q92 | ERROR | prediction: execute: execute: no such column: C.client_id | |||
| q94 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 10, Col: 6. FROM district)) -- Get the account id of the oldest female client with the lowest average salary [4mSELECT[0m MAX(A11) - MIN(A11) AS Gap FROM district WHERE A11 <> (SELECT A11 FROM district WHERE A11 IS NULL) | |||
| q95 | ERROR | prediction: execute: execute: no such column: account_id | |||
| q98 | ERROR | prediction: execute: execute: ambiguous column name: account_id | |||
| q99 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 4, Col: 29. amount FROM loan WHERE duration > 12 AND strftime('%Y', date) = '1993' JOIN account ON loan.account[4m\[0m_id = account.account\_id ORDER BY amount DESC LIMIT 1; | |||
| q100 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 3, Col: 27. SELECT COUNT(*) FROM client c JOIN account a ON c.client[4m\[0m_id = a.client\_id WHERE c.gender = 'F' AND c.birth\_date < '1950-01-01' AND c.district\_id = ( SELE | |||
| q112 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 3, Col: 33. SELECT district.A2 FROM district JOIN client ON district.district[4m\[0m_id = client.district\_id WHERE client.gender = 'F' AND client.birth\_date = '1976-01-29'; | |||
| q115 | ERROR | prediction: statement: the text does not parse: Expecting ). Line 1, Col: 23. SELECT (COUNT(c.client[4m\[0m_id) FILTER (WHERE c.gender = 'M')) * 100.0 / COUNT(c.client\_id) AS percentage\_of\_male\_clients F | |||
| q125 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 4, Col: 85. district, loan WHERE loan.status = 'D' AND loan.account_id = trans.account_id AND district.district[4m\[0m_id = account.district\_id GROUP BY district.district\_id; | |||
| q128 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 3, Col: 27. A2 as District, COUNT(*) as Number_of_Female_Account_Holders FROM client JOIN disp ON client.client[4m\[0m_id = disp.client\_id JOIN account ON disp.account\_id = account.account\_id WHERE gender = 'F' GROU | |||
| q138 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 6, Col: 27. trict.A15) FROM district WHERE district.A2 = 'branch_name')) AND client.gender = 'M'; Just replace [4m'branch\_name'[0m with the actual name of the branch where the second-highest number of crimes were committed in 1995 | |||
| q149 | ERROR | prediction: execute: execute: no such column: account.type | |||
| q159 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 15. SELECT t.trans[4m\[0m_id, t.date, t.type, t.operation, t.amount, t.balance FROM trans t JOIN disp d ON t.account\_id = d. | |||
| q168 | ERROR | prediction: statement: the text does not parse: Expecting ). Line 1, Col: 30. SELECT 100.0 * COUNT(c.client[4m\[0m_id) / ( SELECT COUNT(c.client\_id) FROM client c JOIN district d ON c.district\_id = d.district\_id | |||
| q169 | ERROR | prediction: execute: execute: no such column: a.client_id | |||
| q186 | ERROR | prediction: statement: the text does not parse: Expecting ). Line 1, Col: 30. SELECT 100.0 * COUNT(c.client[4m\[0m_id) / ( SELECT COUNT(DISTINCT o.account\_id) FROM `order` o JOIN account a ON o.account\_id = a.acc | |||
| q189 | ERROR | prediction: execute: execute: no such column: A11 | |||
| q194 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 21. SELECT client.client[4m\[0m_id, client.birth\_date, JULIANDAY(CURRENT\_DATE) - JULIANDAY(client.birth\_date) AS age FROM client |
A question whose statement could not be run wrote no directory, so it is a row here and has no page: the summary holds the side that stopped, the step it stopped at and the engine's own message.
The JSON this page was rendered from
- summary.json attestql/audit/summary/2