Delete Azure Blob Files Only In a Folder

If you are looking for a command to delete Azure blob files in a folder with one command, here is the solution;

Continue…

A solution to make PowerBI Direct Query Reports parametrised in CI/CD pipeline

PowerBI REST APIs are very limited with restrictions for specific usages. One of the restriction is that you cannot update parameters on your PowerBI dataset via APIs if you’re using Direct Query. It is a problem if you want to have a master template PowerBI report and create reports based on the master for each different project with different parameters.

Continue…

Export Bunq account statements for UK visa application via Bunq’s APIs

Six months ago, I applied to UK visa together with a solicitor. My solicitor asked me to provide daily bank statements for the last three months to prove that my account balance never down below asked level by Home Office. At that time, I was using Bunq and had a chat with Bunq support, and I was told that it is not possible via application unless I export statements day by day. That means I had to do some actions for 90 times. Thank god, I’m a developer and Bunq provides API that can automate it.

Continue…

Export MS SQL tables to Parquet Files

The following code exports MS SQL tables to Parquet files via PySpark. It can be used in tables that do not have an indexed column with the numerical type (int, float, etc.).

Continue…

Partition Parquet File by Date

The pyspark script below can split one single big parquet file into small parquet files based on date column.

Continue…