Pulling data from PDFs is this developers’ challenge and nightmare. Here is an approach using Microsoft Excel and “Data Query” (or is it “Power Query”?)
This is a piece for advanced VBA coders. You can find the workbook here.
The first trick is to get a list of the “pages” or tables in the PDF. That’s important because we’ll need to code custom parsing routines. They’ll only work with specific PDF pages or tables.
To explain my techniques I’ve created a project around extracting data from the U.S. Department of Labor weekly unemployment reports.
They look like this
Do you find this joke funny?
Classifying mathematical problems as linear or nonlinear is like classifying the Universe into bananas and non-bananas.
I laughed. But would most people?
How about this?
Mathematics is made of 50 percent formulas, 50 percent proofs, and 50 percent imagination.
Not only would non-mathematicians stare at you blankly, they might think you’re a dimwit. Most people believe math is about universal truths, not some squishy imagination.
How about these jokes?
Why do they lock gas station bathrooms? Are they afraid someone will clean them?
A car hit an elderly man. The paramedic says…
For most people, photography starts with a JPG, a computer file of compressed pixel data, or, for video, an MP4 file.
The Internet is awash with experts who explain how to make cinema-quality movies from cameras or phones purchased at BestBuy. They don’t explain that these image files are simplifications, degraded versions, of what the camera saw. Many of them imply that one can get the same results from these compressed files that one could get from the original sensor data, which is called RAW. In some cases, this is true. In others, false.
In this story, I hope to…
A good friend, cheering on my YouTube endeavors, told me to watch How To With John Wilson on HBO.
I’m trying to get this YouTube thing out of my system. I wouldn’t bother reading this unless you, too, are similarly afflicted.
In his third episode, How to Improve Your Memory, he begins to interview a winner of memory contests. She mentions the memory palace technique. I would have let the camera roll for 10 hours; it’s a deep subject. Instead, he moves onto a small group of oddballs focused on mistaken memories.
Problem #1: The deeper into my subject I…
There are many reasons why people are moving out of California, New York and Massachusetts. Putting aside the increasing levels of the homeless and Covid-19, some people will move to low-tax states in the coming years to avoid paying off debt their states accumulated to pay unemployment benefits they didn’t save for.
Do you have some complex charts and slides you, or your client, want posted on Twitter, LinkedIn, Medium, FB or almost any web page?
Here’s output from this project. Tracking Covid and Temperature in Massachusetts.
I want a one-stop solution for robust internet API calls, data extraction, transformations, charting, SFTP uploads, etc. So I used Excel VBA and command-line apps to create animated GIFs. The following is a template that can be quickly modified…
The carnage begins now, in September. Each week, from here out, tens of thousands of Massachusetts unemployed will lose their UI benefits. The bulge of people who lost their jobs in late March, 2020 are about to exhaust their benefits. (I focus on Massachusetts numbers). For this analysis I’m excluding the 13-weeks Pandemic Emergency Unemployment Compensation (PEUC) extension because it is unknown how many will be automatically enrolled.
Whenever benefits end, why is everyone so certain the Federal government can, or will, save them? …
Doesn’t the pandemic make you think of all that’s gone wrong? I was thinking about skyscraper residential buildings. Do people really want to live in a sterile white room, high above a city, with an elevator the only means of escape? How won’t they end up dilapidated and bankrupt? Oops, I’m drifting into stuff I’m usually wrong about…
Then my thoughts drifted to cruise ships. I’ve never seen their attraction either. I wouldn’t have predicted they’d end up a multi-billion dollar industry.
Maybe someone can make a board game out of this. What is society’s 7 deadly sins to you?
The wealthy of New England won’t take pay cuts, or ask for additional taxes to help the poorer neighborhoods. That may be a long-term tragic mistake.
When I suggest to anyone in Cambridge that home prices may not, in the future, double every five years they say the same thing, “As long as Harvard is in Cambridge we’ll be okay.” That’s a very short memory. Most of Cambridge was in the dumps 20 years ago and racial tensions were high. What about the 364 years between 1636 to 2000?
The Boston area hasn’t become desirable because of Harvard alone. Just…
The other shoe has not dropped. There will be no recovery…
Until rental evictions are lifted we won’t know how many people will be living on the street — then the economy won’t grow until they’re settled.
Until consumer credit loosens there will be no consumer-led economic growth.
Until mortgage forbearance is removed we won’t know how much residential or commercial property debt will survive — then the economy won’t grow until the borrowers complete their bankruptcies and real estate re-prices.
Until the Fed stops micro-managing market liquidity, the invisible hand of capitalism will remain shackled to political favoritism and…