#PYDATA

Posted on December 4, 2022 in python · pydata · visualization · til

TIL how to use fontawesome markers with folium. Read More

Posted on December 1, 2022 in pydata · python · meetup

Forecasts crave a rating that reflects the forecast's quality in the context of what is possible in theory and what is reasonable to expect in practice. Read More

Posted on November 16, 2022 in python · sql · snowflake · pydata

Snowflake offers different ways to access and call python from within their compute infrastructure. This post will show how to access python in user defined functions, via stored procedures and in snowpark. Read More

Posted on September 27, 2020 in python · sql · pydata · azure

Using SQLAlchemy to create openrowset common table expressions for Azure Synapse SQL-on-Demand Read More

Posted on May 25, 2020 in python · parquet · pydata · pandas · azure

Inspired by Uwe Korns post on DuckDB this post shows how to use Azure Synapse SQL-on-Demand to query parquet files with T-SQL on a serverless cloud infrastructure. Read More

Posted on May 25, 2020 in python · sql · pydata · azure

Azure Synapse SQL-on-Demand offers a web client, the desktop version Azure Data studio and odbc access with turbodbc to query parquet files in the Azure Data Lake. Read More

Posted on February 1, 2020 in python · pydata · azure · parquet

Last summer Microsoft has rebranded the Azure Kusto Query engine as Azure Data Explorer. While it does not support fully elastic scaling, it at least allows to scale up and out a cluster via an API or the Azure portal to adapt to different workloads. It also offers parquet support out of the box which made me spend some time to look into it. Read More

Posted on January 19, 2020 in python · pydata · parquet · arrow · pandas

Apache Parquet is a columnar file format to work with gigabytes of data. Reading and writing parquet files is efficiently exposed to python with pyarrow. Additional statistics allow clients to use predicate pushdown to only read subsets of data to reduce I/O. Organizing data by column allows for better compression, as data is more homogeneous. Better compression also reduces the bandwidth required to read the input. Read More

Posted on January 11, 2020 in python · pydata

Microsoft has released a beta version of the python client azure-storage-file-datalake for the Azure Data Lake Storage Gen 2 service with support for hierarchical namespaces. Read More

Posted on April 19, 2019 in exasol · azure · pydata · talk

Exasol on Microsoft Azure – automatic deployment in less than 30 minutes Read More

Posted on October 8, 2015 in python · pydata · spark

Cloudera Kudu is a distributed storage engine for fast data analytics. The python api is in alpha stage but already usable. Read More

Posted on August 4, 2015 in python · pydata · spark · talk

Apache Spark is a computational engine for large-scale data processing. PySpark exposes the Spark programming model to Python. It defines an API for Resilient Distributed Datasets (RDDs) and the DataFrame API. Read More

Posted on July 10, 2015 in python · pydata · spark · talk

This Talk from PyData 2015 Berlin gives an overview of the PySpark Data Frame API. Read More