What Are Logs and Data Analytics? Explained in One Diagram

Log analysis is essentially about compressing raw records layer by layer into a handful of actionable numbers. Most teams stall at the second layer—they have lo…

Insights:What Are Logs and Data Analytics? Explained in One Diagram

Five-Layer Architecture

Visit / Error / Business Events
        ↓ Collection
      Raw Logs
        ↓ Storage
      Queryable Archives
        ↓ Cleaning
      Deduplicated Records
        ↓ Aggregation
      Metrics (PV/UV/Conversion/Error Rate)
        ↓ Decision
     Actions (Modify Page/Fix Config/Add Protection)

Each layer compresses information, leaving only the numbers that can drive action.

Collection Layer

  • Access logs: who requested what and when.
  • Error logs: where failures occurred and their causes.
  • Business events: registration, form submission, checkout, etc.

The key to collection is uniform formatting and parsable fields.

Storage Layer

  • Small sites: store directly as files, split by day.
  • Medium sites: ingest into databases or log services.
  • Key requirement: fast queries by time range.

Cleaning Layer

  • Remove crawlers and internal traffic.
  • Deduplicate records (retries, replays).
  • Unify timestamps and time zones.

Logs left uncleaned will produce systematically inflated metrics.

Aggregation Layer

Metric Meaning
PV/UV Traffic volume
Bounce & Depth Content usefulness
Conversion rate Proportion of forms/submissions
Error rate System stability
Traffic sources Channel effectiveness

Decision Layer

  • Metric anomaly → Identify the specific page or time period.
  • Pinpoint the cause → Formulate an action item.
  • Execute the action → Review metric changes.

Without a judgment layer, the first four layers are all cost.

Common Breakpoints

  • No business-event logging; only access records are captured.
  • Logs exist but field meanings are unclear.
  • Someone reads reports, but no one does comparisons.
  • Focus only on traffic, not error rates.

Boundaries

This article covers principles and does not involve specific tool configuration. For selection and commands, see other articles in the same cluster.

Related reading: