What’s the deal with log files?

Not All Media Data Is Created Equal

September 26, 2019

Rich data is one of the key benefits of digital advertising.  It enables both precise targeting and detailed performance reports.  Unlike other digital media, programmatic generates impression-level transaction data, in the form of log files, which are very granular in detail.

What exactly is a “Log file”?

Every time a programmatic impression is served, a detailed digital receipt is created. Each log receipt contains roughly 200 data points, including details about the device and browser, the timestamp, viewability and, perhaps most importantly, the exact price of the impression. Logs also reveal information about the supply chain, including all the vendors, the fees they charged, and what sites the ads ran on.  Read more about log files here.

Each log receipt contains roughly 200 data pointsincluding the exact price of the impression.

That’s a lot of data.  Do you really need that level of detail? 

Traditional media transactions are tracked with basic aggregate data stored in flat files, typically CSV.  Search and Social media platforms offer transaction and performance data via API, and again this data is aggregated and reveals the averages (e.g total impressions, average CPM, CPC, etc.)  Unlike programmatic logs, these aggregate data sources offer nothing about the individual impressions.  

This aggregate data is great for producing monthly KPI reports, and providing both the client and the agency a high level snapshot of how their campaigns performed.  But this aggregated data does not allow for full analysis.

When do you need Log-level data?

Be sure to capture and normalize Log-level data if you want to do any of the following:

  • See a full accounting of all fees charged by your agencies and vendors
  • Understand every component of your supply chain, including all vendors and publishers
  • Calculate the exact cost of fraudulent and non-viewable impressions
  • Perform a thorough programmatic audit
  • Apply any kind of AI analysis to the data 
  • Generate benchmarks and baselines 

What do I need in order to collect and analyze Logs? 

  • First you need to be sure your agency contract preserves your rights to the log data.  While it’s becoming less common, some advertisers are surprised to learn they don’t have rights to log files. Advertisers paid for the media – they own and should retain all resulting data.
  • Next, tell your vendors you want them, so they stop purging them.  It’s important to note that Log files are not available from vendors after 60-90 days. (In some cases even less time)  They are purged periodically because they are costly for the vendors to store. In the past, vendors used to charge nominal fees to provide log data, but most now provide them free of charge upon request.
  • Set-up a data warehouse optimized for storing log data at scale, and capable of normalizing the data for full impression-level analysis.  Simply storing the log files in an S3 bucket or similar Cloud storage has little value.  
  • Make sure the data is clean.  Log files have a lot of fields, and they should all be populated.
  • Establish a standard Naming Convention.  Each vendor may label log fields differently.  Make sure like fields have the same naming convention, so normalization is achieved, and monitor new data to insure it adheres to the common taxonomy.
  • Last, leverage a flexible reporting and visualization tool to properly analyze the data and identify issues and opportunities for optimization.

When is aggregate data sufficient?

Aggregate data is great for getting a bird’s eye view of your campaigns, and monitoring your performance versus KPIs.  It’s comparable to the annual summary you receive from your credit card company or your stock broker: you can see what you spent in total, and the breakdown by category.  That’s useful to get a sense of numbers, and to insure nothing is too far outside acceptable thresholds.

But the devil is in the details.  Did someone use your card without your knowledge?  Did one stock trade account for 90% of your losses?  Or gains? This kind of detail (and insight) is only gleaned from the itemized statements, and close scrutiny.  The same holds true of your media data.

Aggregate data provides guidance, but log-level data, at the impression level, enables full visibility and control.

The Takeaway.  

Paid media, and specifically digital media, generates mountains of data. Aggregating the data makes it easier to summarize and consume. But as with all data, the more granularity you have, the more intelligence you can gain, and the better you can optimize your spending.  

Bottom line: if you invest in programmatic media you should always retain and analyze your log files. With AD/FIN’s Media Intelligence Platform, you can manage all your paid media data, both log-level and aggregated, in one system, under one dashboard.

-Matt Deloca, CRO


If you’re interested in learning more about leveraging media data and log files for advanced reporting sign up for our webinar on Beyond Reporting: The Importance of Centralizing Your Media Data Oct 17, 2019 at 1pm! Register Here.

Want to Learn More or Schedule a Demo?

AD/FIN Headquarters

25 West 31st Street 4th floor
New York, NY 10001

(646) 762-0270


Follow Us