Leveraging the nTop nDPI for Application Visibility within Sycope/nProbe integration

Gaining insight into Layer 7 application is one of the crucial advantages for network monitoring in areas of performance and security. The ability to know what kind of application is used by users within our network helps us to comply with various policies and standards, either internal or imposed by directives, and enhance our troubleshooting capabilities.

Thanks to the close understanding of NetFlow protocol developer’s team of Sycope provide close integration with nTop's nProbe solution, that could provide visibility into Layer 7.

nProbe, a solution developed by nTop company, is a network probe that, based on the copy of network traffic, can provide information for details such as retransmission rates, application and network latency, TCP and TLS connection details and Layer 7 application identification. nProbe full fill this feature with nTop's nDPI engine, that can efficiently identify over 300 applications. This information are later encapsulated into NetFlow and send to Sycope system for processing. In this brief article, I will present steps necessary to enable nProbe's nDPI Layer 7 information processing within Sycope system.

As a first step, we need to ensure that nProbe has field %L7_PROTO included into it's NetFlow template and is using IPFIX/ NetFlow v10 to export information to Sycope. Since Sycope v2.6 it is mandatory configuration to support all integration features.

Next, using Sycope Discovery Mode, we analyse the template received from nProbe and use a simple wizard to add a previously unknown field with ID 118 to our database.

We have to provide a name and display name for the new field. Also, providing the description is good practice. Then we should specify field type as "long" in order to store information from Field ID 119 in a most efficient way for our database. We should also consider enabling option Indexed field. This option will optimize the field in case of using it within search queries and speeds up aggregation process. We have to choose Traffic Profile associated with nProbe and define deduplication strategy. In this particular case, “Last(any)” or “First(Any)” will fit our needs.

‍

In the second step of the wizard, we should enable custom aggregation. Thanks to this, Sycope will automatically aggregate statistical data extracted from NetFlow for unique application IDs, which will optimize and speed up some long-term searches. Keep in mind that the system will require reboot after this action.

The information gathered from the new NetFlow field is accessible as a number. We need to represent it in the form of the application’s name, understandable for human analytic. To do that, we need to extract the applications list used by nDPI implemented into our instance of nProbe. We can access this list using command nprobe -H. Save this information as CSV file.

‍

Next, we need to prepare a lookup within Sycope system that will match ID extracted from NetFlow field to application’s name. Using lookup creator located under Settings - Configuration - Mapping - Lookups section, define lookup type as "CSV file", provide a name and brief description. Then use "Edit CSV" and import file prepared earlier. Save your work and switch to Mappers section.

Using creator, define new mapper by providing Display name, brief description, match method and item source as “From lookup”. In Value field and Name field, we need to specify names of header elements of your csv file.

I suggest using mapper based on lookup instead of embedded into mapper because of additional information that nProbes applications list provides (severity, category). If we use csv embedded into mapper, then all additional info won't be available for other Sycope’ features.

The last step is to create an additional database field that will display the mapper result. Go to Settings - Configuration - Objects - Fields section and use creator to define field settings. Define mandatory settings and in Source section point to the NetFlow field that you configured with Sycope Discovery Mode. As a first source, use "NetFlow", as a second source use custom aggregation that was configured during the second stage of Sycope Discovery Mode wizard. Lastly, choose your mapper from the drop-down list under Mappers. Save your work and check the outcome in the Raw Data.

From this point nDPI application definitions may serve as additional insight into your analysis, filters and dimension fields for your widgets.

‍

Author of this article: Jan Rześny, Support and Deployment Engineer, Passus S.A

‍

‍