Report Layouts

What information should a Usage Report contain? The answer will likely vary depending on who will be reviewing it, and what information they hope to acquire from it. Report Reviewers will likely want a bird's eye view of what occurred during the report period. The network administrator will want more details to help him configure policies.

The report engine of the Security Appliance project uses sections to generate reports containing data at various levels. Want a bird's eye view? Create a Report Layout that only contains the Graphs and Genres sections. Want lots of detail? Create a Report Layout that contains domain name sections.

Section Descriptions

Every available section is summarized below. Click on each summary to see an example report containing only that section. These sections are grouped together to form Report Layouts.

Applications Section

The Applications Section shows activity by the App / App Type used.

Graphs Section

The Graphs Section contains several bar charts to summarize the Ratings and Filter (allow/block) Actions that occurred during the report period.

Activity Through the Day Bar Chart

This chart indicates how usage activity varies throughout the hours of the day. The viewer can normally expect to see a curve that corresponds to normal waking hours of the day.

Also, the graph shows that system generated traffic is usually much higher user-generated traffic through the browser.

Activity Through the Day002002004004006006008008001k1k1.2k1.2k1.4k1.4k1.6k1.6k1am2am3am4am5am6am7am8am9am10am11am12pm1pm2pm3pm4pm5pm6pm7pm8pm9pm10pm11pm12am020.552884615384613272.276184646584061am047.59615384615383272.276184646584062am074.63942307692308272.276184646584063am0101.6826923076923272.276184646584064am0128.72596153846152272.276184646584065am0155.7692307692308272.276184646584066am2182.81250000000003272.11962310058527am499209.85576923076928233.214078919873268am215236.89903846153848255.445818451708659am329263.9423076923077246.5218103297747210am268290.9855769230769251.2969374827393711am136318.02884615384613261.629999518662912pm142345.07211538461536261.160314880666361pm166372.1153846153846259.28157632868032pm150399.15865384615387260.5340686966713pm239426.20192307692304253.567079899722564pm292453.24519230769226249.41819893075335pm31480.28846153846155269.8494806836026pm0507.3317307692307272.276184646584067pm0534.375272.276184646584068pm0561.4182692307692272.276184646584069pm0588.4615384615385272.2761846465840610pm0615.5048076923076272.2761846465840611pm0642.5480769230769272.2761846465840612am032.45192307692307272.276184646584061am059.49519230769229272.276184646584062am086.53846153846155272.276184646584063am0113.58173076923077272.276184646584064am0140.625272.276184646584065am0167.66826923076925272.276184646584066am79194.71153846153848266.09200357962987am1.508k221.75480769230774154.228778963458078am918248.79807692307693200.41443503311619am878275.84134615384613203.5456659530929410am1.226k302.88461538461536176.3039569492946411am1.192k329.9278846153846178.9655032312749412pm1.138k356.9711538461538183.192664973243641pm1.705k384.01442307692304138.807466682572252pm952411.0576923076923197.75288875113583pm1.371k438.1009615384615164.95324486437874pm1.256k465.1442307692307173.9555337593125pm267492.1875251.37521825573886pm139519.2307692307692261.39515719966467pm0546.2740384615385272.276184646584068pm0573.3173076923076272.276184646584069pm0600.3605769230769272.2761846465840610pm0627.4038461538461272.2761846465840611pm0654.4471153846154272.2761846465840612amActivity Through the DayHit CountHuman ActivitySystem Activity

Category & Search Ratings Bar Charts

The data for these chart comes from the Human Activity data which focuses on traffic that likely originated from a human user's browser session, and Searches, which are search terms used on common search engines, e-commerce stores and stock photo sites.

These charts give a quick overview of the general tone of the usage/activity during the report period.

User Activity by Rating00101020203030404050506060707080809090http://localhost:8000/reporter/reporter/logviewer/loglines/page/1/?start_date=2016-04-01+00%3A00&userips= Activity by Ratingsilt-49%pebble-21%base-20%sand-7%stone-4%

Parent Categories Section

The Parent Categories Section groups Parent and Child Categories together to give a quick overview of how the activity was classified during the report period.

Parent Categories Bar Chart

The Parent Category Bar Chart is fairly self-explanatory as it merely compares the relative popularity of the various parent categories. The Tech / Computing category will frequently be one of the highest scoring parent categories as it includes several "base" categories that serve as a kind of "catch-all" for APIs and programmatic communications.

Parent Categories002244668810101212141416161818202022222424262628283030Financial Services31.8777.0673076923077111.0Tech/Computing21.493125.50961538461539145.75196712606504E-Commerce11.81173.95192307692312178.17976877217544Business/Industrial7.881222.3942307692308191.33776037266782Automotive6.54270.8365384615385195.82869107672997Science5.2319.2788461538462200.3162728391784Entertainment/Arts4.553367.72115384615387202.48303806328596Internal/Systems/Apis2.796416.1634615384616208.3671284786754Education2.334464.6057692307693209.91433950423595Social Networking1.017513.048076923077214.3248956095677News0.924561.4903846153848214.63634717964808Real Estate0.67609.9326923076925215.48697834954504Parent Categories% PopularityFinancial Services-33%Tech/Computing-22%E-Commerce-12%Business/Industrial-8%Automotive-7%Science-5%Entertainment/Arts-5%Internal/Systems/APIs-3%Education-2%Social Networking-1%News-1%Real Estate-1%

Parent Category Details Tables

This section is the meat and potatos of the report engine. The Parent Categories from the barchart above are shown in expanded detail here. The following information is summarized in this section:

Search Term Section

User searches from common search engines, e-commerce, stock photo and reference sites are displayed here. This section gives some of the clearest data on the activity of users. The list of search terms is not necessarily comprehensive. If there is a common site where search terms are not being extracted, please file a support ticket to investigate.

Searches are perhaps the most dynamic of all web page content. The same term can be scored differently, depending on many factors totally determined by the algorithm used by the search engine. User activity, popular culture, times of the day among other things may affect search results, and by extension, the classifying of those search results. For this reason, the same term be rated multiple ways in the same report period.

Media Views

Media Views from hosting service companies such as Youtube and Vimeo are shown here, including Media title, hosting service and Rating color.

Only Allow views are reported. If the view attempt was Blocked, it won't appear in the report.

Domains Accessed By System Activity

This section shows domains that were probably accessed by systems or software running in the background. The domains in this list may not be recognizable by the users on the network.

The domains are colorized by Category Ratings, according to the ratio of times they were scored in a given Rating.

System Activity

The activity reported in this section was likely system-generated. Software updates for printers, cameras, computers and other devices. Other system-related traffic may also apply such as keep-alive updates between clients and servers, etc.

Erased Content

This section shows content that was blocked inside of other pages, while the main page loaded. Think of the paper catalogs from the postal system. If you ever tore out an indecent page, or used a sharpie on an unwanted picture, that's the kind of blocking activity that this section represents.

The domains are colorized by Category Ratings, according to the ratio of times they were scored in a given Rating. Ads are not colorized, however, because those blocks aren't worthy of an administrative review. As a percentage of overall blocking activity, Ads should represent the vast majority of all filter blocking incidents.

Erased Activity

This is perhaps the least significant section. None of the domains in this section are likely to have been the direct result of user-generated activity. Ad-blocked domains do not have a colored background.

In addition to ads, much of the blocks that are reported in this section are cleanup. Images, files and other content assets may have been blocked, while the main page loaded.

Shredded Content

This section shows content from domains that were blocked every time an access was attempted. Think of the paper catalogs from the postal system. If you ever received a paper catalog and just through the entire thing into the trash can, that's the kind of blocking activity that this section represents.

The domains are colorized by Category Ratings, according to the ratio of times they were scored in a given Rating.

Shredded Activity

Domains that appear in this section were blocked every time access was attempted. Because every request was denied, it's not possible to determine with any chance of accuracy whether the activity was user- generated or system-generated.

If this section contains many domains, it usually indicates undesireable activity. Virus-infected systems, for example can trigger requests that will end up here. Or, if this section is large, perhaps a human user is responsible.

The Complete List

That is the complete list of sections for usage reports. Mix and match from these sections to create Report Layouts that will contain the information of greatest interest. The Security Appliance project includes a list of common Report Layouts for immediate use.