{"id":541,"date":"2025-07-17T18:47:41","date_gmt":"2025-07-17T18:47:41","guid":{"rendered":"https:\/\/sreschool.com\/blog\/?p=541"},"modified":"2025-07-17T19:11:10","modified_gmt":"2025-07-17T19:11:10","slug":"networkout-complete-guide","status":"publish","type":"post","link":"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/","title":{"rendered":"NetworkOut Complete Guide"},"content":{"rendered":"\n

To see exactly what traffic is being served through your ens5<\/code> interface<\/strong> (the interface AWS bills for \u201cNetworkOut\u201d), you have two primary options:<\/p>\n\n\n\n

\n\n\n\n

Option 1: Use VPC Flow Logs (Recommended for AWS)<\/strong><\/h2>\n\n\n\n

VPC Flow Logs<\/strong> will capture all incoming and outgoing network traffic at the ENI (network interface) level\u2014so you\u2019ll see everything going in\/out through ens5<\/code>.<\/p>\n\n\n\n

How to Enable and Analyze VPC Flow Logs:<\/strong><\/h3>\n\n\n\n
    \n
  1. Enable VPC Flow Logs:<\/strong>\n
      \n
    • Go to your AWS VPC Dashboard > \u201cYour VPCs\u201d > Select your VPC.<\/li>\n\n\n\n
    • Choose Actions \u2192 Create Flow Log<\/strong>.<\/li>\n\n\n\n
    • For \u201cFilter\u201d, select All<\/strong>.<\/li>\n\n\n\n
    • For \u201cDestination\u201d, choose an S3 bucket or CloudWatch Logs.<\/li>\n\n\n\n
    • Create the flow log.<\/li>\n<\/ul>\n<\/li>\n\n\n\n
    • Download\/View Flow Logs<\/strong> from S3 or CloudWatch.<\/li>\n\n\n\n
    • Analyze Flows for Outbound Traffic:<\/strong>
      • Each flow log line will include: source IP, destination IP, port, bytes transferred, etc.<\/li>
      • You can filter for outbound traffic<\/strong> (srcaddr<\/code> = your EC2\u2019s private IP, or srcport<\/code> = 443\/80).<\/li><\/ul>Example command (after downloading a log file):<\/strong>grep \"172.31.34.212\" flowlog.log | awk '$4 == \"172.31.34.212\" {print $5, $7, $10}' | sort | uniq -c | sort -nr | head -20<\/code>\n
          \n
        • This shows destination IP, destination port, and bytes sent.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n
          \n\n\n\n

          Option 2: Use ss<\/code> or netstat<\/code> on the Server<\/strong><\/h2>\n\n\n\n
            \n
          • ss -tpn<\/code> (like you used) shows active connections<\/strong>, not historical bandwidth.<\/li>\n\n\n\n
          • To see live traffic<\/strong> on ens5<\/code>: sudo iftop -i ens5<\/code>\n
              \n
            • This will give you a real-time, interactive view of traffic in\/out of ens5<\/code>.<\/li>\n\n\n\n
            • You\u2019ll see which remote IPs are sending\/receiving the most data right now.<\/li>\n<\/ul>\n<\/li>\n\n\n\n
            • Or, use nload<\/code>: sudo nload ens5<\/code>\n
                \n
              • Shows live upload\/download speeds for that interface.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
                \n\n\n\n

                Option 3: Use Apache\/Nginx Logs<\/strong><\/h2>\n\n\n\n
                  \n
                • As you\u2019ve already done, the logs show what content is being served<\/strong>, but don\u2019t directly tie to interfaces.<\/li>\n\n\n\n
                • All public traffic<\/strong> to your EC2 web server goes out through ens5<\/code> unless you have multiple network cards or special routing.<\/li>\n<\/ul>\n\n\n\n
                  \n\n\n\n

                  Summary Table<\/strong><\/h2>\n\n\n\n
                  Tool<\/th>Shows historic?<\/th>Shows live?<\/th>Shows content?<\/th>AWS integrated?<\/th><\/tr><\/thead>
                  VPC Flow Logs<\/td>\u2705<\/td>\u274c<\/td>Only IP\/Port<\/td>\u2705<\/td><\/tr>
                  iftop\/nload<\/td>\u274c<\/td>\u2705<\/td>No (just IPs)<\/td>\u274c<\/td><\/tr>
                  Apache Logs<\/td>\u2705<\/td>\u274c<\/td>Yes (URLs)<\/td>\u274c<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
                  \n\n\n\n

                  Typical Workflow to Check \u201cens5\u201d Traffic<\/strong><\/h2>\n\n\n\n
                    \n
                  1. Enable VPC Flow Logs<\/strong> for full history and billing-level analysis.<\/li>\n\n\n\n
                  2. Use iftop<\/code> for live monitoring<\/strong> on ens5<\/code> (great for troubleshooting spikes).<\/li>\n\n\n\n
                  3. Correlate with web server logs<\/strong> for content details.<\/li>\n<\/ol>\n\n\n\n
                    \n\n\n\n

                    <\/h2>\n\n\n\n
                    ### MAGICAL COMMAND\t\t\ngrep -o 'GET [^ ]\\+ ' access_log | sort | uniq -c\t\t\ngrep -o 'GET [^ ]\\+ ' access_log | sort | uniq -c | sort\t\t\ngrep -o 'POST [^ ]\\+ ' access_log | sort | uniq -c | sort\t\t\n\t\t\n\t\t\ngrep -o 'GET [^ ]\\+ ' ssl_request_log | sort | uniq -c | sort\t\t\ngrep -o 'POST [^ ]\\+ ' ssl_request_log | sort | uniq -c | sort\t\t\n\t\t\n------\t\t\ngrep -o \"[0-9]\\+\\.[0-9]\\+\\.[0-9]\\+\\.[0-9]\\+\" access_log | sort | uniq -c\t\t\nawk '{ print $1}' access_log | sort | uniq -c | sort -nr | head -n 10\t\t\n-----\t\t\nawk '{ print $3 }' ssl_request_log | sort | uniq -c | sort -nr | head -n 10\t\t\n\t\t\n\t\t\nFind the Top IPs by Bandwidth Used\t\t\nawk '{ip=$3; size=$NF} {bytes[ip]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i],i}' ssl_request_log | sort -nr | head -20\t\t\n\t\t\n\t\t\nFind the Top URLs by Bandwidth Used\t\t\nawk '{url=$7; size=$NF} {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i],i}' ssl_request_log | sort -nr | head -20\t\t\n\t\t\nFind Top Downloaded Files\t\t\nawk '{url=$7; size=$NF} \/\\.pdf|\\.jpg|\\.png|\\.zip|\\.mp4|\\.mov\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i],i}' ssl_request_log | sort -nr | head -20\t\t\n\t\t\nFind Top User Agents (bots\/browsers) by Requests\t\t\nawk -F\\\" '{print $6}' ssl_request_log | sort | uniq -c | sort -nr | head -20\n\nOnly Images, JS, CSS, PDFs, ZIPs, Videos\nawk '{url=$7; size=$NF} \/\\.jpg|\\.jpeg|\\.png|\\.gif|\\.svg|\\.webp|\\.ico|\\.js|\\.css|\\.pdf|\\.zip|\\.mp4|\\.mov|\\.avi\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -30\n\nTop Consumers by File Extension (e.g., images only)\nawk '{url=$7; size=$NF} \/\\.jpg|\\.jpeg|\\.png|\\.gif|\\.svg|\\.webp|\\.ico\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -20\n\nJavaScript Only\nawk '{url=$7; size=$NF} \/\\.pdf\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -20\n\n\t\t<\/code><\/pre>\n\n\n\n
                    Here\u2019s a ready-to-use list of Linux commands<\/strong> you can run on your log files (like ssl_request_log<\/code> or access_log<\/code>) to identify which images, JS, PDFs, and other big assets are consuming the most bandwidth<\/strong>.<\/p>\n\n\n\n
                    \n\n\n\n
                    1. Top Resource URLs by Total Bytes Sent (all file types)<\/strong><\/h2>\n\n\n\n
                    awk '{url=$7; size=$NF} {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -30\n<\/code><\/pre>\n\n\n\n
                      \n
                    • What it does:<\/strong> Lists top 30 URLs by total data sent (in bytes).<\/li>\n<\/ul>\n\n\n\n
                      \n\n\n\n
                      2. Only Images, JS, CSS, PDFs, ZIPs, Videos<\/strong><\/h2>\n\n\n\n
                      Change the file extensions as needed.<\/p>\n\n\n\n
                      awk '{url=$7; size=$NF} \/\\.jpg|\\.jpeg|\\.png|\\.gif|\\.svg|\\.webp|\\.ico|\\.js|\\.css|\\.pdf|\\.zip|\\.mp4|\\.mov|\\.avi\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -30\n<\/code><\/pre>\n\n\n\n
                        \n
                      • What it does:<\/strong> Sums up bandwidth for only assets ending in .jpg<\/code>, .png<\/code>, .js<\/code>, .css<\/code>, .pdf<\/code>, .zip<\/code>, .mp4<\/code>, .mov<\/code>, etc.<\/li>\n\n\n\n
                      • You can add\/remove file types by editing the regex.<\/strong><\/li>\n<\/ul>\n\n\n\n
                        \n\n\n\n
                        3. Top Consumers by File Extension (e.g., images only)<\/strong><\/h2>\n\n\n\n
                        Images Only<\/h3>\n\n\n\n
                        awk '{url=$7; size=$NF} \/\\.jpg|\\.jpeg|\\.png|\\.gif|\\.svg|\\.webp|\\.ico\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -20\n<\/code><\/pre>\n\n\n\n
                        JavaScript Only<\/h3>\n\n\n\n
                        awk '{url=$7; size=$NF} \/\\.js\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -20\n<\/code><\/pre>\n\n\n\n
                        PDF Only<\/h3>\n\n\n\n
                        awk '{url=$7; size=$NF} \/\\.pdf\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -20\n<\/code><\/pre>\n\n\n\n
                        Video Only (mp4, mov, avi)<\/h3>\n\n\n\n
                        awk '{url=$7; size=$NF} \/\\.mp4|\\.mov|\\.avi\/ {bytes[url]+=(size~\/^[0-9]+$\/?size:0)} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -20\n<\/code><\/pre>\n\n\n\n
                        \n\n\n\n
                        4. Top Consumers by Client IP and Asset<\/strong><\/h2>\n\n\n\n
                        Which IP is requesting which large resource the most:<\/strong><\/p>\n\n\n\n
                        awk '{ip=$3; url=$7; size=$NF} {combo=ip\" \"url} (size~\/^[0-9]+$\/) && (url ~ \/\\.jpg|\\.png|\\.js|\\.css|\\.pdf|\\.zip|\\.mp4|\\.mov|\\.avi\/) {bytes[combo]+=size} END {for(i in bytes) print bytes[i], i}' ssl_request_log | sort -nr | head -30\n<\/code><\/pre>\n\n\n\n
                          \n
                        • Shows which IP and URL combinations are sending the most data.<\/li>\n<\/ul>\n\n\n\n
                          \n\n\n\n
                          5. Top 404\/Errors (Optional, helps in asset wastage detection)<\/strong><\/h2>\n\n\n\n
                          awk '$9 ~ \/^404$\/ {print $7}' ssl_request_log | sort | uniq -c | sort -nr | head -20\n<\/code><\/pre>\n\n\n\n
                            \n
                          • (If your log format has status code at field $9<\/code>)<\/li>\n<\/ul>\n\n\n\n
                            \n\n\n\n
                            Tips<\/strong><\/h2>\n\n\n\n
                              \n
                            • Replace ssl_request_log<\/code> with your actual log file name if different.<\/li>\n\n\n\n
                            • Adjust field numbers if your log format is not the default combined log format.<\/li>\n\n\n\n
                            • For daily\/weekly data, filter with grep \"2025-07-18\"<\/code> or similar before the awk<\/code>.<\/li>\n<\/ul>\n\n\n\n
                              \n\n\n\n
                              <\/p>\n","protected":false},"excerpt":{"rendered":"
                              To see exactly what traffic is being served through your ens5 interface (the interface AWS bills for \u201cNetworkOut\u201d), you have […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-541","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\nNetworkOut Complete Guide - SRE School<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"NetworkOut Complete Guide - SRE School\" \/>\n<meta property=\"og:description\" content=\"To see exactly what traffic is being served through your ens5 interface (the interface AWS bills for \u201cNetworkOut\u201d), you have […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/\" \/>\n<meta property=\"og:site_name\" content=\"SRE School\" \/>\n<meta property=\"article:published_time\" content=\"2025-07-17T18:47:41+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-17T19:11:10+00:00\" \/>\n<meta name=\"author\" content=\"Rajesh Kumar\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Rajesh Kumar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/\",\"url\":\"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/\",\"name\":\"NetworkOut Complete Guide - SRE School\",\"isPartOf\":{\"@id\":\"https:\/\/sreschool.com\/blog\/#website\"},\"datePublished\":\"2025-07-17T18:47:41+00:00\",\"dateModified\":\"2025-07-17T19:11:10+00:00\",\"author\":{\"@id\":\"https:\/\/sreschool.com\/blog\/#\/schema\/person\/0ffe446f77bb2589992dbe3a7f417201\"},\"breadcrumb\":{\"@id\":\"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/#breadcrumb\"},\"inLanguage\":\"en\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/sreschool.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"NetworkOut Complete Guide\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/sreschool.com\/blog\/#website\",\"url\":\"https:\/\/sreschool.com\/blog\/\",\"name\":\"SRESchool\",\"description\":\"Master SRE. Build Resilient Systems. Lead the Future of Reliability\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/sreschool.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/sreschool.com\/blog\/#\/schema\/person\/0ffe446f77bb2589992dbe3a7f417201\",\"name\":\"Rajesh Kumar\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en\",\"@id\":\"https:\/\/sreschool.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g\",\"caption\":\"Rajesh Kumar\"},\"sameAs\":[\"http:\/\/sreschool.com\/blog\"],\"url\":\"https:\/\/sreschool.com\/blog\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"NetworkOut Complete Guide - SRE School","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/","og_locale":"en_US","og_type":"article","og_title":"NetworkOut Complete Guide - SRE School","og_description":"To see exactly what traffic is being served through your ens5 interface (the interface AWS bills for \u201cNetworkOut\u201d), you have […]","og_url":"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/","og_site_name":"SRE School","article_published_time":"2025-07-17T18:47:41+00:00","article_modified_time":"2025-07-17T19:11:10+00:00","author":"Rajesh Kumar","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Rajesh Kumar","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/","url":"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/","name":"NetworkOut Complete Guide - SRE School","isPartOf":{"@id":"https:\/\/sreschool.com\/blog\/#website"},"datePublished":"2025-07-17T18:47:41+00:00","dateModified":"2025-07-17T19:11:10+00:00","author":{"@id":"https:\/\/sreschool.com\/blog\/#\/schema\/person\/0ffe446f77bb2589992dbe3a7f417201"},"breadcrumb":{"@id":"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/#breadcrumb"},"inLanguage":"en","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sreschool.com\/blog\/networkout-complete-guide\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sreschool.com\/blog\/networkout-complete-guide\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sreschool.com\/blog\/"},{"@type":"ListItem","position":2,"name":"NetworkOut Complete Guide"}]},{"@type":"WebSite","@id":"https:\/\/sreschool.com\/blog\/#website","url":"https:\/\/sreschool.com\/blog\/","name":"SRESchool","description":"Master SRE. Build Resilient Systems. Lead the Future of Reliability","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sreschool.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en"},{"@type":"Person","@id":"https:\/\/sreschool.com\/blog\/#\/schema\/person\/0ffe446f77bb2589992dbe3a7f417201","name":"Rajesh Kumar","image":{"@type":"ImageObject","inLanguage":"en","@id":"https:\/\/sreschool.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g","caption":"Rajesh Kumar"},"sameAs":["http:\/\/sreschool.com\/blog"],"url":"https:\/\/sreschool.com\/blog\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/posts\/541","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/comments?post=541"}],"version-history":[{"count":3,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/posts\/541\/revisions"}],"predecessor-version":[{"id":545,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/posts\/541\/revisions\/545"}],"wp:attachment":[{"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/media?parent=541"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/categories?post=541"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/tags?post=541"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}