Log filtering SolarWinds uses cookies on its websites to make your online experience easier and better. By using our website, you consent to our use of cookies. For more information on cookies, see our Cookie Policy. Continue Visit SolarWinds.com Documentation Contact Us Customer Portal Toggle navigation Academy SOLARWINDS ACADEMY CLASSES GUIDED CURRICULUM ELEARNING CERTIFICATION SOLARWINDS ACADEMY The SolarWinds Academy offers education resources to learn more about your product. The curriculum provides a comprehensive understanding of our portfolio of products through virtual classrooms, eLearning videos, and professional certification. See What's Offered AVAILABLE RESOURCES Virtual Classrooms Calendar eLearning Video Index SolarWinds Certified Professional Program VIRTUAL CLASSROOMS Attend virtual classes on your product and a wide array of topics with live instructor sessions or watch on-demand videos to help you get the most out of your purchase. Find a Class Open Sessions and Popular Classes General Office Hours Orion Platform Network Performance Monitor NetFlow Traffic Analyzer See All IP Address Manager Network Configuration Manager Server & Application Monitor Virtualization Manager GUIDED CURRICULUM Whether learning a newly-purchased SolarWinds product or finding information to optimize the software you already own, we have guided product training paths that help get customers up to speed quickly. View Suggested Paths ELEARNING On-demand videos on installation, optimization, and troubleshooting. See All Videos Popular Videos Upgrading Isn't as Daunting as You May Think Upgrading Your Orion Platform Deployment Using Microsoft Azure Upgrading From the Orion Platform 2016.1 to 2019.4 Don't Let the Gotchas Get You How to Install NPM and Other Orion Platform Products Upgrading the Orion Platform See All Videos Navigating the Web Console Prepare a SAM Installation Installing Server & Application Monitor How to Install SEM on VMware Customer Success with the SolarWinds Support Community New job, New to SolarWinds? SOLARWINDS CERTIFIED PROFESSIONAL PROGRAM Become a SolarWinds Certified Professional to demonstrate you have the technical expertise to effectively set up, use, and maintain SolarWinds’ products. Learn More STUDY AIDS Access Rights Manager Architecture and Design Database Performance Analyzer Diagnostics NetFlow Traffic Analyzer Network Configuration Manager Network Performance Monitor Server & Application Monitor ONBOARDING & UPGRADING NEW TO SOLARWINDS ORION ASSISTANCE PROGRAM UPGRADE RESOURCE CENTER SUPPORT OFFERINGS SMARTSTART WHAT’S NEW UPGRADE RESOURCE CENTER See helpful resources, answers to frequently asked questions, available assistance options, and product-specific details to make your upgrade go quickly and smoothly. Visit the Upgrade Resource Center PRODUCT-SPECIFIC UPGRADE RESOURCES Network Performance Monitor NetFlow Traffic Analyzer Network Configuration Manager Server & Application Monitor Storage Resource Monitor Virtualization Manager Web Performance Monitor Log Analyzer ORION ASSISTANCE PROGRAM This program connects you with professional consulting resources who are experienced with the Orion Platform and its products. These services are provided at no additional charge for customers who were/are running one of the Orion Platform versions affected by SUNBURST or SUPERNOVA. Learn More SUPPORT OFFERINGS Our Customer Support plans provide assistance to install, upgrade, and troubleshoot your product. Choose what best fits your environment and organization, and let us help you get the most out of your purchase. We support all our products, 24/7/365. Learn more AVAILABLE PROGRAMS Professional Premier Premier Enterprise SMARTSTART Our SmartStart programs help you install and configure or upgrade your product. Get assistance from SolarWinds’ technical support experts with our Onboarding and Upgrading options. We also offer a self-led program for Network Performance Monitor (NPM) and Server & Application Monitor (SAM) if you need help doing it yourself. Learn more AVAILABLE PROGRAMS SmartStart for Onboarding SmartStart for Upgrading SmartStart Self-Led for NPM and SAM WHAT’S NEW AT SOLARWINDS Find the latest release notes, system requirements, and links to upgrade your product. Learn More NEW TO SOLARWINDS You just bought your first product. Now what? Find out more about how to get the most out of your purchase. From installation and configuration to training and support, we've got you covered. Learn More Support Offerings PREMIER SUPPORT SMARTSTART WORKING WITH SUPPORT PREMIER SUPPORT We offer paid Customer Support programs to assist you with installation, upgrading and troubleshooting. Choose what best fits your environment and budget to get the most out of your software. Get priority call queuing and escalation to an advanced team of support specialist. AVAILABLE PROGRAMS Premier Support Premier Enterprise Support SMARTSTART Our SmartStart paid programs are intended help you install and configure or upgrade your product. You’ll be assisted by SolarWinds’ technical support experts who are dedicated to quickly and efficiently help you with getting up and running or moving to the latest version of your product. AVAILABLE PROGRAMS SmartStart for Onboarding SmartStart for Upgrading Working with Support WORKING WITH SUPPORT A glossary of support availability, tips, contact info, and customer success resources. We're here to help. Learn More PRODUCTS NETWORK MANAGEMENT SYSTEMS MANAGEMENT DATABASE MANAGEMENT IT SECURITY IT SERVICE MANAGEMENT APPLICATION MANAGEMENT DOCUMENTATION NETWORK MANAGEMENT Orion Platform Network Performance Monitor NetFlow Traffic Analyzer IP Address Manager Network Configuration Manager Engineer's Toolset View All Network Management Products Network Topology Mapper User Device Tracker VoIP Network Quality Manager Log Analyzer Enterprise Operations Console Your SolarWinds products come with a secret weapon. Award-winning, instructor-led classes, eLearning videos, and certifications. Find a Class SYSTEMS MANAGEMENT Server & Application Monitor Virtualization Manager Storage Resource Monitor Web Performance Monitor Server Configuration Monitor Backup View All Systems Management Products Your SolarWinds products come with a secret weapon. Award-winning, instructor-led classes, eLearning videos, and certifications. Find a Class IT SECURITY Security Event Manager Access Rights Manager Serv-U Managed File Transfer Server Serv-U FTP Server Patch Manager Identity Monitor View All IT Security Products Your SolarWinds products come with a secret weapon. Award-winning, instructor-led classes, eLearning videos, and certifications. Find a Class DATABASE MANAGEMENT Database Performance Analyzer Database Performance Monitor View All Database Management Products Your SolarWinds products come with a secret weapon. Award-winning, instructor-led classes, eLearning videos, and certifications. Find a Class IT SERVICE MANAGEMENT Dameware Remote Everywhere Dameware Remote Support Dameware Mini Remote Control Service Desk Web Help Desk View All IT Service Management Products Kiwi Syslog Server Kiwi CatTools ipMonitor Mobile Admin Your SolarWinds products come with a secret weapon. Award-winning, instructor-led classes, eLearning videos, and certifications. Find a Class APPLICATION MANAGEMENT AppOptics Pingdom Papertrail Loggly View All Application Management Products Your SolarWinds products come with a secret weapon. Award-winning, instructor-led classes, eLearning videos, and certifications. Find a Class COMMUNITY THWACK® ORANGE MATTER LOGICALREAD THWACK® Over 150,000 users—get help, be heard, improve your product skills Visit THWACK AVAILABLE PROGRAMS SolarWinds Lab THWACK Tuesday Tips (TTT) THWACKcamp™ 2020 On-demand Orange Matter Practical advice on managing IT infrastructure from up-and-coming industry voices and well-known tech leaders View Orange Matter LogicalRead Blog Articles, code, and a community of database experts Read the Blog SUBMIT A TICKET Academy SOLARWINDS ACADEMY See What's Offered Virtual Classrooms Calendar eLearning Video Index SolarWinds Certified Professional Program CLASSES Find a Class General Office Hours Orion Platform Network Performance Monitor NetFlow Traffic Analyzer See All IP Address Manager Network Configuration Manager Server & Application Monitor Virtualization Manager GUIDED CURRICULUM View Suggested Paths ELEARNING See All Videos Upgrading Isn't as Daunting as You May Think Upgrading Your Orion Platform Deployment Using Microsoft Azure Upgrading From the Orion Platform 2016.1 to 2019.4 Don't Let the Gotchas Get You How to Install NPM and Other Orion Platform Products Upgrading the Orion Platform See All Videos Navigating the Web Console Prepare a SAM Installation Installing Server & Application Monitor How to Install SEM on VMware Customer Success with the SolarWinds Support Community New job, New to SolarWinds? CERTIFICATION Learn More Access Rights Manager Architecture and Design Database Performance Analyzer Diagnostics NetFlow Traffic Analyzer Network Configuration Manager Network Performance Monitor Server & Application Monitor ONBOARDING & UPGRADING NEW TO SOLARWINDS Learn More ORION ASSISTANCE PROGRAM Learn More UPGRADE RESOURCE CENTER Visit the Upgrade Resource Center Network Performance Monitor NetFlow Traffic Analyzer Network Configuration Manager Server & Application Monitor Storage Resource Monitor Virtualization Manager Web Performance Monitor Log Analyzer SUPPORT OFFERINGS Learn More Professional Premier Premier Enterprise SMARTSTART Learn more SmartStart for Onboarding SmartStart for Upgrading SmartStart Self-Led for NPM and SAM WHAT’S NEW Learn More Support Offerings PREMIER SUPPORT Premier Support Premier Enterprise Support SMARTSTART SmartStart for Onboarding SmartStart for Upgrading Working with Support WORKING WITH SUPPORT Learn More PRODUCTS NETWORK MANAGEMENT Orion Platform Network Performance Monitor NetFlow Traffic Analyzer IP Address Manager Network Configuration Manager Engineer's Toolset View All Network Management Products Network Topology Mapper User Device Tracker VoIP Network Quality Manager Log Analyzer Enterprise Operations Console SYSTEMS MANAGEMENT Server & Application Monitor Virtualization Manager Storage Resource Monitor Web Performance Monitor Server Configuration Monitor Backup View All Systems Management Products IT SECURITY Security Event Manager Access Rights Manager Serv-U Managed File Transfer Server Serv-U FTP Server Patch Manager Identity Monitor View All IT Security Products DATABASE MANAGEMENT Database Performance Analyzer Database Performance Monitor View All Database Management Products IT SERVICE MANAGEMENT Dameware Remote Everywhere Dameware Remote Support Dameware Mini Remote Control Service Desk Web Help Desk View All IT Service Management Products Kiwi Syslog Server Kiwi CatTools ipMonitor Mobile Admin APPLICATION MANAGEMENT AppOptics Pingdom Papertrail Loggly View All Application Management Products DOCUMENTATION COMMUNITY THWACK® Visit THWACK ORANGE MATTER View Orange Matter LOGICALREAD Read the Blog SUBMIT A TICKET Documentation forPapertrail Log filtering Introduction Papertrail can filter incoming log messages that match one or more strings or regular expressions (regex) of your choosing. Log filtering is included with all Papertrail accounts and filtered messages don't consume log data transfer. Filters are specific to a destination, so different environments, systems, or apps can have their own settings. Papertrail's log filter is an additional tool. The sending client or app can still filter logs, like with the remote_syslog2 exclude_patterns option or by changing an app’s log settings. These filters are independent of any Papertrail filter. Trying to find a log message in Papertrail's log viewer? This page covers how to drop log messages, not how to search for them. Visit Search syntax instead. Search queries can't be used directly as log filters, but they can usually be translated. Read on to learn how to translate a search query (e.g. host:abc AND program:123) to a regular expression (e.g. ^abc 123:). Quick start Log in to Papertrail and click Settings, then click Filter logs under the usage bar. Select Log Filters for the desired destination. (This step isn’t necessary for Heroku add-on accounts.) In the log filtering settings, enter the case-sensitive string that matches messages Papertrail should ignore. More. Example uses Ignore noise. "Noise" could be requests from monitoring agents, requests for static assets, requests that succeeded and did not modify a resource, or any other log messages which are unlikely to be useful. Control log verbosity. Drop unnecessary messages from services that you do not have access or ability to change. For example, a closed-source app, a managed service, or a system with strict change control. Environment-specific log configuration. For example, retain everything in staging and development but silence certain messages in production, or vice versa. Team-wide control. Let anyone on a team see and change the filtering settings, without needing to understand and modify a config file. Infrastructure-wide control. Create a single regular expression that reflects your own logging preferences, then apply it to log streams from many systems and apps at once. Examples Here are a few common uses. See Setup for complete docs. Filtering all occurrences of 3 messages This will drop all messages containing any of the 3 strings. All matches are case-sensitive. Filtering one program or log file from one sender Imagine you have one program generating log messages that you don't want. Filter all messages from the program mongod on the system db-server-42 using the regex: Copy ^db-server-42 mongod The ^ indicates that the match must happen from the start of a log message. The sender name (in this example, db-server-42) is the name as shown on the Dashboard. Substrings Regexes automatically match substrings (unless anchors in the regex specify position, as above). That is, these three expressions are identical: Copy cron .*cron.* cron.* and will all match any string containing cron, with or without any leading or following characters. Including .* before or after a typical filter rule is unnecessary. Filtering ("disabling") multiple senders Imagine you have two sending systems which are temporarily generating an undesirable torrent of log messages that you don't want. Filter all messages from the senders system-a and system-b: Copy ^(system-a|system-b) Or filter only messages from noisy-file.log on these two senders: Copy ^(system-a|system-b) noisy-file.log (If just one system is going crazy, consider temporarily muting it instead.) Setup Decide what to filter Visit Events and browse the full log stream with all log messages. Decide which messages you want to filter. Each line in the log stream is scanned individually, so multiline or related messages can't be filtered with a single expression. Create filter Click Settings, then click Filter logs under the usage bar. Then, select a log destination’s Log Filters (not necessary for Heroku users). In one of the boxes in the Log Filters area, enter a string or construct a regex that matches each of the messages Papertrail should filter. For example, to filter all log messages containing debug, enter debug as the filter and choose String. Use the Add and Save buttons to create more filters and save the changes. Log filters can only be created on account-specific destinations. Logs from senders using a public IP to send to port 514 cannot be filtered. Test regular expression When constructing a regex to filter messages, we recommend using Rubular with Ruby version 2.0.0 selected for testing. This isn’t exactly the same regex engine that Papertrail uses, but it’s a close approximation. Paste the filter expression created above, then copy a sample log message of each message type that should be matched. The expression matches against everything shown in the Papertrail viewer except for the timestamp, so include the sender name, program name, a colon, and then the message (as shown in the Your test string: input box below). For example: Since this regex matches the log message shown, Papertrail would silently discard the message. Finally, paste a log message that should not match. If it still matches, refine the regex. The characters .|()[]{}\^$+?* have special meaning when using a regex and need to be escaped by placing a \ before them. To match log messages containing GET a.b.c type=json, use a filter string that escapes each special character: Copy GET a\.b\.c type=json Papertrail doesn't allow "lookaround" ((?!, (?=, (?<) regular expression elements because they have unpredictable performance. Try writing an alternative regex, altering the client’s log configuration, or adding identifiable content to the problem messages. Contact us for further help with advanced filtering. Advanced Filtering multiple messages A more complex example would match multiple messages or only messages from certain senders or apps. For example, suppose that these two messages serve no operational purpose: Copy www42 httpd: 127.0.0.1 - "GET / HTTP/1.0" 200 3 and Copy util2 kernel: nf_conntrack: automatic helper assignment is deprecated and it will be removed soon. Use the iptables CT target to attach helpers instead. Find the portion of the log that occurs in all such messages. Here we'll use 127.0.0.1 - "GET / HTTP/1.0" 200 (a successful HTTP request from the Web server itself) and nf_conntrack: automatic helper assignment is deprecated (a warning which could be repeated). The following would filter all successful web requests (any HTTP status 200-299) and the warning: Filtering colorized messages If your logs contain hidden ANSI color codes, these can interfere with filtering. Suppose that your logs colorize the log level (DEBUG, INFO, WARN, ERROR) that occurs immediately after the program name: and you want to filter out INFO and DEBUG messages from app1. To match an ANSI escape code, use the regex (\e\[[0-9;]*m)?. For example, to filter the log lines above, use an expression such as: Copy app1: (\e\[[0-9;]*m)?(INFO|DEBUG)(\e\[[0-9;]*m)? The regex Copy /app1: (INFO|DEBUG)/ would not match, even though it appears to align with the displayed character string. Papertrail's standard system and program colors are not applied using ANSI color codes, so don't require any special filtering. Filtering by sender Papertrail matches your regular expression against the complete log message as it is formatted in the viewer. This allows you to include the name of the sender and/or program (or a substring of them) as part of the filter. Using the examples above, to filter each message from only the system shown in the example, use a filter like: The ^ indicates that the match must be at the very start of the log. The sender name is the same display name shown on the Papertrail dashboard. Note: if you use the sender name in a filter and then edit the sender name, the filter will need to be updated as well. Filter by default Papertrail's default policy is to process messages it receives, which means that the filter string is deciding which messages are ignored. While there's currently no support for an inverse filter (default behavior of ignoring log messages), these two workarounds often accomplish the same behavior: Filter them locally. All common loggers (rsyslog, syslog-ng, remote_syslog2) can filter by message contents. Pick the top few message types and have papertrail filter them. Often this can get close to the same result. Here's an example. In most cases this is simply: Copy string from one|string from the other|repeat for more messages If you have a filtering requirement that Papertrail can't serve well, please tell us. The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation. We’re Geekbuilt.® Developed by network and systems engineers who know what it takes to manage today's dynamic IT environments, SolarWinds has a deep connection to the IT community. The result? IT management products that are effective, accessible, and easy to use. COMPANY INVESTORS CAREER CENTER RESOURCE CENTER EMAIL PREFERENCE CENTER FOR CUSTOMERS FOR GOVERNMENT GDPR RESOURCE CENTER SOLARWINDS TRUST CENTER Legal Documents Privacy California Privacy Rights Security Information Documentation & Uninstall Information Sitemap © 2021 SolarWinds Worldwide, LLC. All rights reserved.