The whole of this run is a release asset
This site holds 0 of the 15 question directories this run wrote; the archive holds every one:
- archive
- minidev-sqlite-mistralai-mixtral-8x7b-instru-4.tar.gz
- size
- 1,075,349 bytes
- sha256
- 7f34c38f4a2fbc7c9cfd7720ee1dcc8af8c27748e47e2bdd6ebe556f7e3703ea
What it counted
| what | how many |
|---|---|
| questions audited | 51 |
| EQUAL | 13 |
| ERROR | 23 |
| NOT_EQUAL | 15 |
| probes fired | 3 |
the probes
| probe | fired on |
|---|---|
| ordering-over-numeric-text | 0 |
| arbitrary-cut | 1 |
| not-a-function-of-the-data | 2 |
| float-aggregate-order | 0 |
| direction-against-question | 0 |
credited by BIRD's own check and NOT_EQUAL here
| what | how many |
|---|---|
| credited by BIRD and NOT_EQUAL here | 2 |
| multiplicity | 2 |
| type | 0 |
| order | 0 |
| truncation | 0 |
| other | 0 |
| the test-suite check answered 1 | 0 |
| the test-suite check answered 0 | 2 |
What the run was made of
- run
- audit-fa26f81a-cce0-4423-814f-6c1c690ef447
- summary format
- attestql/audit/summary/2
- question file
- data/questions/mini_dev_sqlite.json sha256:4ba5fa8de55856222f484d380d2ba872b380bf79d825de70478e2120cb0fc43b | https://bird-bench.oss-cn-beijing.aliyuncs.com/minidev.zip (sha256 cc48ba16838204e4e214512030cb572eeb5f7bcdd999bae4b9b6ff12ec13b92f, downloaded 2026-09-07), member minidev/MINIDEV/mini_dev_sqlite.json, 2024-06-19
- questions in the file
- 500
- prediction file
- data/preds-sqlite/predict_mini_dev_mistralai-mixtral-8x7b-instru-4_sqlite.json sha256:53cdf9142e3f2e49941eb7b50ebbea90361b2c5b46de37d5ba1c244187d643a2 | https://raw.githubusercontent.com/bird-bench/mini_dev/b3d4bcbbae9a96934ad812551eb400c7a3b23c12/llm/exp_result/sql_output_kg/predict_mini_dev_mistralai-mixtral-8x7b-instru-4_sqlite.json, 2024-06-19
- statements read
- 500
- keyed by
- position
- data file
- data/minidev/dev_databases/european_football_2/european_football_2.sqlite sha256:f72f5c9b990371417288a67b27a02e601ef904d45525c2c9f7931fb01001c536 | minidev.zip (sha256 cc48ba16838204e4e214512030cb572eeb5f7bcdd999bae4b9b6ff12ec13b92f), member minidev/MINIDEV/dev_databases, european_football_2/european_football_2.sqlite, 2024-06-13
- server
- SQLite 3.53.4 | file=/private/tmp/attestql-runs/data/minidev/dev_databases/european_football_2/european_football_2.sqlite | size=597754880
- database role
- file
- engine
- sqlite
- parser
- validator audit:sqlglot-sqlite-parse, sqlglot 30.18.0, dialect sqlite
- serialization
- attestql/audit/2
- statement timeout
- 30 s
- fixture digest depth
- counts
- schema digest
- sha256:954a8a8d6524994d3b40baa9fb2a91c922f170e93b000994be908ae47378e5e7
- data as of
- 2026-09-08T05:04:45.603103+00:00 the instant the run started
- shuffled copies
- prepared
- shuffle seed
- 1
- experimental probe
- off
The session the run was made in
what the engine reported, 9 settings
- sqlite_version
- 3.53.4
- encoding
- UTF-8
- reverse_unordered_selects
- 0
- query_only
- 1
- journal_mode
- delete
- data_version
- 2
- compile_options
- ATOMIC_INTRINSICS=1,COMPILER=clang-21.0.0,DEFAULT_AUTOVACUUM,DEFAULT_CACHE_SIZE=-2000,DEFAULT_FILE_FORMAT=4,DEFAULT_JOURNAL_SIZE_LIMIT=-1,DEFAULT_MMAP_SIZE=0,DEFAULT_PAGE_SIZE=4096,DEFAULT_PCACHE_INITSZ=20,DEFAULT_RECURSIVE_TRIGGERS,DEFAULT_SECTOR_SIZE=4096,DEFAULT_SYNCHRONOUS=2,DEFAULT_WAL_AUTOCHECKPOINT=1000,DEFAULT_WAL_SYNCHRONOUS=2,DEFAULT_WORKER_THREADS=0,DIRECT_OVERFLOW_READ,ENABLE_API_ARMOR,ENABLE_COLUMN_METADATA,ENABLE_DBSTAT_VTAB,ENABLE_FTS3,ENABLE_FTS3_PARENTHESIS,ENABLE_FTS5,ENABLE_GEOPOLY,ENABLE_MATH_FUNCTIONS,ENABLE_MEMORY_MANAGEMENT,ENABLE_PERCENTILE,ENABLE_PREUPDATE_HOOK,ENABLE_RTREE,ENABLE_SESSION,ENABLE_STAT4,ENABLE_UNLOCK_NOTIFY,MALLOC_SOFT_LIMIT=1024,MAX_ATTACHED=10,MAX_COLUMN=2000,MAX_COMPOUND_SELECT=500,MAX_DEFAULT_PAGE_SIZE=8192,MAX_EXPR_DEPTH=1000,MAX_FUNCTION_ARG=1000,MAX_LENGTH=1000000000,MAX_LIKE_PATTERN_LENGTH=50000,MAX_MMAP_SIZE=0x7fff0000,MAX_PAGE_COUNT=0xfffffffe,MAX_PAGE_SIZE=65536,MAX_SQL_LENGTH=1000000000,MAX_TRIGGER_DEPTH=1000,MAX_VARIABLE_NUMBER=250000,MAX_VDBE_OP=250000000,MAX_WORKER_THREADS=8,MUTEX_PTHREADS,SYSTEM_MALLOC,TEMP_STORE=1,THREADSAFE=1,USE_URI
- collation_list
- RTRIM,NOCASE,BINARY
- case_sensitive_like
- 0
What the run states about itself
- ids the question file states twice
- 137, 138
- prediction positions not compared
- 487, 488
The questions
| question | rule | verdict | class | probes | what was asked |
|---|---|---|---|---|---|
| q1028 | ERROR | prediction: execute: execute: no such column: League.name | |||
| q1029 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 17. SELECT Team.team[4m\[0m_short\_name, Team\_Attributes.buildUpPlaySpeed FROM Team JOIN Team\_Attributes ON Team.team\_api\_i | |||
| q1036 | ERROR | prediction: execute: execute: no such column: team_long_name | |||
| q1037 | ERROR | prediction: execute: execute: no such function: MULTIPLY | |||
| q1040 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 21. SELECT Player.player[4m\[0m_name FROM Player JOIN Player\_Attributes ON Player.player\_api\_id = Player\_Attributes.player\_api | |||
| q1048 | ERROR | prediction: execute: execute: no such column: player_name | |||
| q1057 | ERROR | prediction: statement: the text does not parse: Expecting ). Line 1, Col: 16. SELECT AVG(home[4m\[0m_team\_goal) FROM Match WHERE country\_id = (SELECT id FROM Country WHERE name = 'Poland') AND seaso | |||
| q1076 | ERROR | prediction: statement: the text does not parse: Expecting ). Line 1, Col: 18. SELECT AVG(Player[4m\[0m_Attributes.ball\_control) - (SELECT AVG(Player\_Attributes.ball\_control) FROM Player\_Attributes W | |||
| q1079 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 14. SELECT player[4m\[0m_api\_id, MAX(height) as TallestHeight FROM Player GROUP BY player\_api\_id; | |||
| q1088 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 14. SELECT player[4m\[0m_name FROM Player\_Attributes JOIN Player ON Player\_Attributes.player\_api\_id = Player.player\_api | |||
| q1094 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 6, Col: 3. ting FROM Player_Attributes WHERE player_name = 'Paulin Puel')), 100)), 2) AS percentage_difference [4m```sql This SQL query calculates the percentage difference between Ariel Borysiuk's overall rating and Paulin Puel's overall rating. It first selects the overall ratings for both players, then calculates the difference, divides it by Paulin Puel's overall rating, multiplies the result by 100, and finally rounds it to two decimal places. ```[0m | |||
| q1098 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 41. SELECT MAX(chanceCreationPassing) as max[4m\[0m_chance\_creation\_passing, chanceCreationPassingClass as passing\_class FROM Team_Attributes JOIN T | |||
| q1102 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 21. SELECT Player.player[4m\[0m_name FROM Player\_Attributes JOIN Player ON Player\_Attributes.player\_api\_id = Player.player\_api | |||
| q1103 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 2, Col: 12. SELECT overall_rating FROM Player[4m\[0m_Attributes WHERE player\_name = 'Aaron Mooy' AND date LIKE '2016-02-04%'; | |||
| q1110 | ERROR | prediction: execute: execute: no such column: Team.league_id | |||
| q1114 | ERROR | prediction: execute: execute: no such column: player_name | |||
| q1116 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 14. SELECT player[4m\[0m_name FROM Player ORDER BY height DESC LIMIT 1; | |||
| q1122 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 1, Col: 14. SELECT Player[4m\[0m_Attributes.player\_api\_id, MAX(overall\_rating) as Max\_Rating FROM Player\_Attributes INNER JOIN | |||
| q1133 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 3, Col: 13. SELECT COUNT(*) FROM Player WHERE player[4m\[0m_name LIKE 'Aaron%' AND strftime('%Y', birthday) > '1990'; | |||
| q1134 | ERROR | prediction: statement: the text does not parse: Invalid expression / Unexpected token. Line 8, Col: 21. er_api_id JOIN Player AS home_player_23 ON Match.home_player_23 = Player.player_api_id; This query [4mcalculates[0m the absolute difference between the jumping scores of players 6 and 23 in the home team. It first j | |||
| q1141 | ERROR | prediction: execute: execute: no such column: buildUpPlaySpeedClass | |||
| q1147 | ERROR | prediction: execute: execute: no such column: Player_Attributes.player_name | |||
| q1148 | ERROR | prediction: execute: execute: no such column: height |
A question whose statement could not be run wrote no directory, so it is a row here and has no page: the summary holds the side that stopped, the step it stopped at and the engine's own message.
The JSON this page was rendered from
- summary.json attestql/audit/summary/2