Is It Possible to Query Parquet Files from Sqlite?
Is It Possible to Use Sqlite for Querying over Parquet Files? Create Table Test Using Parquet('Test. Parquet'); Select * from Test; 0 2 Answers Use Pandas to...
Is it possible to use SQLite for querying over Parquet files?
CREATE TABLE test USING parquet('test.parquet');
SELECT * from test;
2 Answers
Use Pandas to import the data first!
import pandas as pd
import sqlite3
parquet_path = 'my-data.parquet'
table_name = 'my_data_table'
query = f"SELECT * from {table_name}"
db_conn = sqlite3.connect(database='/tmp/my.db')
df_parquet = pd.read_parquet(parquet_path)
num_rows_inserted = df_parquet.to_sql(table_name, db_conn, index=False)
cursor = db_conn.execute(query)
If your requirement is to query both SQLite data and Parquet data using lightweight and embeddable tool, you can also consider DuckDB with SQLite scanner extension.
Robert Thorne covers electric vehicle innovations, autonomous driving systems, global mobility trends, and automotive engineering developments.
