Some good pointers on data cleansing which also doubles as a good checklist of things to think about at the start of a data project, at least on the data engineering side …
uv: the last word in Python package management?
You are a consultant starting a new project. What do you need to think about? …
Around 2010 UBS released a somewhat hardcore dresscode in Switzerland, for members of the private bank. The tone may be somewhat prim but it…
Let's eliminate unnecessary meetings, prioritize direct communication, avoid jargon, and use common sense to enhance productivity and efficiency …
I include the below--with redaction to protect the innocent--for nostalgia value and also by way of reference. This email went around most of the IDB team of a leading European investment bank …
To prevent your AirPods from switching automatically between your iPhone and Mac, adjust the connection settings on each device to "When Last Connected" …
Running Python on Windows may seem impractical, but if needed (e.g., for non-headless Chrome webdriver), install from python.org, use `cmd` to manage versions with `py --list-paths`, set up a virtual environment with `py -V:3.13 -m venv venv`, and activate it using `.\venv\Scripts\activate.bat` …
Atlassian's "health monitor" offers a structured team check-in process, which may occasionally replace traditional sprint retrospectives by focusing on team dynamics and continuous improvement …
For a quick version of Caravan's spicy turmeric latte, combine heated oat milk with 4 tablespoons of Minor Figures Barista Chai Concentrate and 1/4 teaspoon turmeric …
This is a pain: neither the latest version of Cursor nor the latest version of WSL extension will actually work. As such I cannot recommend this …
Guidelines for presentation formatting, e.g. title capitalization differences between the US and Europe …
Business cases often involve a combination of technology, industry sector, and specific use cases, with their core value derived from quantifying financial benefits, such as time savings and efficiency improvements, which can be capitalised by an appropriate cost of capital …
To prevent Safari from automatically unzipping downloaded files, go to Preferences/Settings, navigate to the General tab, and uncheck "Open 'safe' files after downloading. …
\"Getting started in your own wood\" by Julian Evans and Will Rolls is often recommended as the first book to read for the novice to silviculture …
Matcha: not just for making tea with …
Cursor is an enhanced version of VSCode with superior LLM functionality when compared to Copilot …
As we age past our twenties, we might notice some weight gain due to lifestyle factors like lack of sleep reducing insulin sensitivity, which can be managed by consuming fewer calories and gradually increasing exercise by 10% weekly to meet fitness goals …
Use the `yt-dlp` command to download YouTube videos in a compact size for devices like iPhones, with subtitles and metadata, in MKV format …
To install a recent version of Go on Pop!_OS 22.04 LTS, add the golang-backports PPA, remove the old golang, install golang-go, and verify with `go version` …
Cloud computing, through data centers accessible over the internet, offers scalable and cost-effective solutions by virtualizing resources, but presents challenges like data security and cost management …
If you need subtitle support, use `makemkv`, otherwise `mkvtoolnix-gui` is sufficient for handling video files …
Instructions for turning off Outlook and Teams alerts and turning on Out of Office on mobile …
To set up LaTeX and Pandoc on MacOS, use brew to install BasicTex and Pandoc, update the TeX Live Manager, install necessary packages, and configure your shell path to convert Markdown files to PDFs with Pandoc …
Traditional questions for the end of Sprint Retro, per Agile: What went well? What did not go well? What did we learn? What changes can we …
Mods from Charm\_ provides a user-friendly and convenient way to use LLMs like ChatGPT in the shell by setting your API key as an environment variable, allowing seamless interaction, prompt-stuffing, and history management with commands for history viewing and piping input directly for quick programming inquiries and examples …
After learning Python, exploring tools like venv, makefile, pytest, pydantic, and SQLAlchemy can enhance productivity by managing environments, automating tasks, simplifying testing, enforcing types, and abstracting database interactions …
Data in a database can be accessed using a shell function in `.bashrc` or `.zshrc` to easily query and display in the terminal …
Coursework involving technology often requires students to submit a YouTube video with timestamps to demonstrate specific features, and if the video exceeds the time limit, it can be sped up using Shotcut and the timestamps adjusted with a Python script …
Tried-and-tested brownie recipe for brownie in a loaf tin which remains a bit gooey in at the centre …
To prevent Spotify from starting automatically, navigate to Settings, then Start-up and window behaviour, and select "No" for the "Open Spotify automatically after you log in to your computer" option …
A staple of science fiction, along with hover-boards and iPads, is the pill that somehow contains the nutrition of a single meal …
Brief coverage below of installing Pop\!_OS 22.04 LTS on a Mac Pro 5,1 from early 2009; or, more correctly, a Mac Pro 4,1 which has had a firmware update …
The Elgato Streamdeck is a popular programmable console with customizable buttons, supporting Windows, MacOS, and Linux (via open-source software), and can be used for arbitrary automation through HTTP requests …
Goodhart and Pradhan argue in _The Great Demographic Reversal_ that the disinflationary trends driven by demographics and globalization from 1990 to 2020 are reversing, leading to a future marked by rising inflation and wages, for which financial markets and policymakers are unprepared …
Plex acts as a locally-hosted streaming service like Netflix, offering multi-platform support, an intuitive interface, metadata retrieval, and features like video transcoding, while requiring specific file organization for optimal performance …
In an ETL process, column order can be set explicitly by including only existing columns in the desired sequence, even if not all specified columns are present in the DataFrame …
The guide explains how to set up pyenv-virtualenv on Ubuntu/Pop!\_OS 22.04 LTS along with jupyter-lab, poetry, and nodejs, including creating a Python virtual environment and making the base kernel available within Jupyter …
In a high inflation scenario, using a buy-now-pay-later option like Klarna's "Pay in 3" can effectively provide a 1% discount on consumer goods due to the depreciating value of money over time, as long as no interest is incurred …
To handle time zones effectively, store timestamps as UTC using Postgres' `TIMESTAMPTZ` type, and convert them to the user's timezone as needed …
Caching data using `functools.cache` enhances performance by returning instant results and minimizing API calls, although mutable types like dictionaries require conversion to immutable types like `frozendict` for caching to work …
To disable unwanted features when using ⌘-space in Spotlight, navigate through Apple → System Preferences → Spotlight …
Pop!\_OS is an Ubuntu-based distribution from System76, known for its user-friendly features and efficient installation process, which can revitalize older devices like a 2015 MacBook Pro …
tl;dr: use Dropbox …
I was puzzled as to why my Garmin Forerunner 735XT stubbornly persisted in showing the wrong time even when it was connected to the Garmin app …
To list git repositories and their remotes on Unix/Linux/MacOS, use a script that searches '.git/config' files and omits Homebrew-related entries …
To change a code block to Python syntax highlighting in VS Code, use regex mode in find/replace to replace empty code block markers with "```python" …
The rename utility, available for MacOS, Linux, and FreeBSD, efficiently uses regex to rename files …
You can easily find out how many films are on your Plex server by running a specific SQL query on the server's database …
Getting photos off iPhone with Image Capture
On MacOS installing a fresh version of miniconda with brew is extremely straightforward …
To explore an unfamiliar codebase, you can use IDEs or the `grep -iIr "string" ./*` command to find variable and function usage, with options to ignore case, skip binary files, and search subdirectories recursively …
To excel in technology, embrace proven methods and prioritize simplicity, as newer isn't always better and increased complexity often leads to inefficiencies and difficulties in maintenance …
A local plaintext archive of web pages of interest, stored in Postgres, with the help of the pinboard.in API …
This code sets default plot sizes to 20x10 cm and cycles through a red, green, blue color scheme in matplotlib …
Feature branching in Git involves creating a `myfeature` branch from `dev`, merging changes back to `dev` without fast-forwarding, and retaining the `myfeature` branch for future use …
For usenet success, maintain multiple block accounts on different backbones, and explore resources like Reddit and Usenet guides for valuable information …
One of the best things you can do early on in your career is to attempt to have candid conversations with more experienced people in your team …
Using ODBC drivers to access Azure from Linux
You use credentials in your (jupyter) notebook or python script and you are about to check it in to a (git) repository. What’s the quickest way …
Getting data from a Pandas DataFrame into a relational DB with minimum effort …
Preferred equity valuation metrics by sector …
Some more on SQL/relational databases
To manage disk space, run Dropbox on one machine with Ubuntu 20.04 using a local USB SSD, then share the folder over a network with NFS, since Dropbox requires local filesystem access and won't work directly with NFS symlinks …
Add this bash/zsh-compatible Pomodoro timer script to your `.bashrc` or `.zshrc` to manage work sessions effectively with periodic breaks …
Data scientists often favour Jupyter notebooks for development, but to prepare code for production environments such as APIs or automation tools, it must be refactored into scripts, validated, and enhanced with functions to address data issues like missing values or duplicate columns …
In Git, it's easy to mistakenly commit with incorrect details, but you can view configurations from different scopes using `git config` commands and rectify the last commit's ID with `git commit --amend --reset-author` …
Basic example(s) of linked lists in C with source code …
get_iplayer, software which allows you download from the BBC iplayer service, is back. There are packages for e.g. MacOS with Brew, FreeBSD …
Access the main menu with `MENU` to choose options like `Run-Matrix` with `1`, `Math` with `F4`, `MAT/VCT` with `F1`, and select matrix/vector sizes using `F1` to `F6`, with `F3` for custom dimensions …
A look at UK equities in 2020 with some anticipation of 2021
The shift from analogue to digital sources and then to networked sources has implications for the seeker of good quality sound reproduction at home …
Harnham's survey reveals that London-based data and analytics roles have a 20% pay premium, salaries vary significantly by seniority and role, with higher-paying positions displaying greater variance …
Addendum September 2022\: Since I wrote this, things have changed considerably and it looks like I will have to completely eat my words …
False positive and false negative rates are important to bear in mind in a mass testing scenario …
To learn more about using Vim or Neovim, explore resources by Tim Pope, Drew Neil, and Ruslan Osipov …
To move a PostgreSQL data directory on FreeBSD from UFS to ZFS, stop the service, use `rsync` to copy the data while preserving permissions, update `rc.conf` and the `postgres` user's home directory, and restart the service …
A friend is going away for a while and will have time to kill and uncertain access to the internet. You have a spare external drive and you …
After experiencing distorted audio from the Squeezebox Touch's SPDIF output, a reboot resolved the issue and SSH access can be enabled for easy resets using default credentials and specific SSH settings …
The author transitioned from using the outdated bashpodder to a custom Go-based solution on GitHub that efficiently downloads, hashes, and annotates podcast MP3 files from RSS feeds with improved metadata management and logging features …
The notes summarize market practices in corporate finance regarding WACC, including its calculation through the cost of equity and debt using CAPM and MPT, the application of target capital structures in DCF analysis, and other uses such as goodwill impairment and assessing economic profit …
‘Swapiness’ on linux can be reduced or increased with sysctl — a higher value will make the system swap out memory more aggressively …
After switching to a Dualit Espress-auto 4 in 1 Coffee and Tea Machine, I discovered that the machine struggles with pressure if the coffee is tamped too tightly and resolved the issue by using more porous AeroPress filters to ensure smoother flow …
How to download activity data from Garmin Connect, process the CSV file in Jupyter Lab, and summarize core exercise metrics—primarily distance and time—by day using Python …
To initialize a repo locally and push to GitHub with 2FA and a personal access token, include the token in the remote URL …
For effective pitches and presentations, focus on being concise, engaging, and repetitive, while planning thoroughly, being authentic and interactive, choosing words carefully, maintaining eye contact, and using visuals to enhance understanding and connection with the audience …
Using ffprobe from the ffmpeg suite is an effective method to extract metadata, such as title, artist, and album details, from audio files …
Running Logitech Media Server on Ubuntu within a bhyve VM on FreeBSD offers a complex yet effective multi-room audio solution, particularly for those committed to LMS and FreeBSD, despite a simpler native Linux installation being available …
Rclone allows access to remote storage providers on unsupported platforms by configuring authorization on a supported system, enabling direct access to services like Dropbox on FreeBSD …
After encountering an error with Gatsby post-update, I resolved it by checking my Node version and executing a series of updates and reinstalls, including upgrading Node and dependencies …
The author recommends a set of preferred digital tools and apps for notes, Markdown editing, Python environments, coding, task management, and source code tracking, emphasizing freemium and open-source options like Bear, Markoff, Anaconda, Jupyter, Visual Studio Code, NeoVim, Trello, and GitHub …
PostgreSQL setup on FreeBSD for access over LAN step-by-step
Here are some excellent resources on integrating Jupyter notebooks with blogging, including conversion tips …
A good look at univariate linear regression
On Linux and MacOS, the desktop is at ~/Desktop, while on Windows, you can use PowerShell to find the desktop path with `[Environment]::GetFolderPath("Desktop")` before running your downloaded .exe file …
If not properly configured, Samba can fall fowl of a couple of security issues: CVE-1999-0519: null session/password netbios access …
Sometimes I forget how to differentiate and use this as a handy guide …
Visual Studio Code, enhanced with the vim plugin, offers a robust editing setup, easily installable via a few command lines …
To set up an Ubuntu system with Nvidia drivers and Miniconda for TensorFlow, ensure your Ubuntu version and GPU compatibility, install necessary Nvidia drivers, reboot, install Miniconda, create a TensorFlow environment, and configure GPU settings for testing …
Editing PDFs on MacOS is straightforward using the Preview app for basic tasks, with Formulate Pro available for advanced needs like adding images, best used after initial edits in Preview …
LinkedIn mobile defaults to excessive push notifications, but you can change this by navigating through your profile picture to Settings, then Communications, and finally Notifications to adjust your preferences …
You can use Python's `subprocess` module to control shell programs and pipe data …
Shane Legg and Marcus Hutter propose a definition of universal intelligence, applicable to humans, animals, and machines, as an agent’s ability to achieve goals across diverse environments, emphasizing the value of simpler solutions and the need to perform well in various complexity levels …
To convert a markdown file to a PDF using Pandoc, you can use a command like `pandoc in.md -o out.pdf`, but to customize the default academic style appearance applied by LaTeX, you should include a YAML header specifying layout, font, and other preferences, and note that MacOS users can install BasicTex for a lighter LaTeX distribution …
To turn off auto-renewing subscriptions on an iOS device, go to Settings > iTunes & App Store, tap your Apple ID, select View Apple ID, tap Subscriptions, choose the subscription, and manage it accordingly …
A high-level reference to Python, primarily for use as an aide-mémoire; it should also be useful for someone who is new to Python but familiar with other languages …
Understanding the shell, a powerful programmable interface integral to Unix-based systems like Linux and MacOS, enhances computer control, automates tasks, and acts as a gateway to more advanced programming, emphasizing careful command usage due to its irreversible actions …
Sometimes I forget how to integrate and use this as a handy guide …