Niels Horn's Blog

Random thoughts, tips & tricks about Slackware-Linux, Lego and Star Wars

Barnyard2 – Solving the “Unknown record type” errors

Barnyard2When experimenting with the new Snort version 2.9.0 I ran into some problems with Barnyard2. It would simply crash with errors like:

ERROR: Unknown record type read: 104
Fatal Error, Quitting..

I started looking for information on the internet (Google is our friend…), and found out that at least I was not alone :)

Similar problems

Basically, I found two types of reports:

  1. some with record types like 2148576734, 1697542248, 3569595041, etc.
  2. others more like the error I received, with the unknown record type being either 104 (like mine) or 105

Error type 1 (apparently random record types)

These errors are caused by reading a log from Snort that is not in the “unified2″ format. Barnyard2 only uses the newer unified2 log format from Snort as input.
The solution is to check your Snort configuration file and change it to use the newer format.

Error type 2 (record types 104 and 105)

The first “solution” I found was removing two parameters in the Snort configuration file for the unified2 log format, mpls_event_types and vlan_event_types.
The default line in the Snort 2.9.0 configuration reads:

output unified2: filename merged.log, limit 128, nostamp, mpls_event_types, vlan_event_types

For Barnyard I already had to remove the “nostamp” parameter, as it prefers to work with the time-stamp extensions.
I do not use the mpls option in Snort, but the vlan ID can be useful…

To get Snort working, I removed the two parameters, but was not satisfied. :(

Finding a real solution – Start examining the source code!

After reading through the source code of Barnyard2 and of the new u2spewfoo utility that comes with Snort 2.9.0, I found out that these record types are from version 2 of the Unified2 record format.
In the Barnyard2 source file unified2.h the definitions can be found:

#define UNIFIED2_EVENT 1
#define UNIFIED2_PACKET 2
#define UNIFIED2_IDS_EVENT 7
#define UNIFIED2_EVENT_EXTENDED 66
#define UNIFIED2_PERFORMANCE 67
#define UNIFIED2_PORTSCAN 68
#define UNIFIED2_IDS_EVENT_IPV6 72
#define UNIFIED2_IDS_EVENT_MPLS 99
#define UNIFIED2_IDS_EVENT_IPV6_MPLS 100

//version 2
#define UNIFIED2_IDS_EVENT_V2 104
#define UNIFIED2_IDS_EVENT_IPV6_V2 105

So, if Barnyard2 knows about the record types, why can’t it process them?
Well, I found that in spi_unified2.c there was this piece of code:

int Unified2ReadRecord(void *sph)
{
        DEBUG_WRAP(DebugMessage(DEBUG_LOG,"Reading record type=%u (%u bytes)\n", ntohl(u2hdr.type), ntohl(u2hdr.length)););

        switch (ntohl(u2hdr.type))
        {
                case UNIFIED2_IDS_EVENT:
                        return Unified2ReadEventRecord(sph);
                        break;
                case UNIFIED2_IDS_EVENT_IPV6:
                        return Unified2ReadEvent6Record(sph);
                        break;
                case UNIFIED2_PACKET:
                        return Unified2ReadPacketRecord(sph);
                        break;
        default:
                        FatalError("Unknown record type read: %u\n", ntohl(u2hdr.type));
            break;
        }

        return -1;
}

This means that it will simply show an error message Unknown record type read: ... when it encounters anything different from type 7, 72 or 2. And that’s exactly what it does in our case :)

This gave me something to start working on…
I first wrote a quick patch to simply recognize and handle record type 104 and sent it to the development team to check if this was the way to go. They answered me that it was, so I completed the patch with record type 105 (which is for ip6) and included the necessary routines for the debugging mode.

This is the resulting patch:

--- barnyard2-1.8_orig/src/input-plugins/spi_unified2.c    2010-03-03 08:03:12.000000000 -0300
+++ barnyard2-1.8/src/input-plugins/spi_unified2.c    2010-10-09 23:31:17.000000000 -0300
@@ -62,11 +62,15 @@

 int Unified2ReadEventRecord(void *);
 int Unified2ReadEvent6Record(void *);
+int Unified2ReadEventRecordV2(void *);
+int Unified2ReadEvent6RecordV2(void *);
 int Unified2ReadPacketRecord(void *);

 void Unified2PrintCommonRecord(Unified2EventCommon *evt);
 void Unified2PrintEventRecord(Unified2Event *);
 void Unified2PrintEvent6Record(Unified2Event6 *evt);
+void Unified2PrintEventRecordV2(Unified2Event_v2 *);
+void Unified2PrintEvent6RecordV2(Unified2Event6_v2 *evt);
 void Unified2PrintPacketRecord(Unified2Packet *);

 /* restart/shutdown functions */
@@ -163,9 +167,15 @@
         case UNIFIED2_IDS_EVENT:
             return Unified2ReadEventRecord(sph);
             break;
+        case UNIFIED2_IDS_EVENT_V2:
+            return Unified2ReadEventRecordV2(sph);
+            break;
         case UNIFIED2_IDS_EVENT_IPV6:
             return Unified2ReadEvent6Record(sph);
             break;
+        case UNIFIED2_IDS_EVENT_IPV6_V2:
+            return Unified2ReadEvent6RecordV2(sph);
+            break;
         case UNIFIED2_PACKET:
             return Unified2ReadPacketRecord(sph);
             break;
@@ -221,6 +231,50 @@
     return -1;
 }

+int Unified2ReadEventRecordV2(void *sph)
+{
+    ssize_t                bytes_read;
+    int                 record_size;
+    Spooler                *spooler = (Spooler *)sph;
+
+    record_size = sizeof(Unified2Event_v2);
+
+    if(!spooler->record.data)
+    {
+        // SnortAlloc will FatalError if memory can't be assigned.
+        spooler->record.data=SnortAlloc(record_size);
+    }
+
+    if (spooler->offset < record_size)
+    {
+        /* in case we don't have it already */
+        bytes_read = read(spooler->fd, spooler->record.data + spooler->offset,
+                    record_size - spooler->offset);
+
+        if(bytes_read == -1)
+        {
+            LogMessage("ERROR: read error: %s\n", strerror(errno));
+            return BARNYARD2_FILE_ERROR;
+        }
+
+        if(bytes_read + spooler->offset != record_size)
+        {
+            spooler->offset += bytes_read;
+            return BARNYARD2_READ_PARTIAL;
+        }
+
+#ifdef DEBUG
+        Unified2PrintEventRecordV2((Unified2Event_v2 *)spooler->record.data);
+#endif
+
+        spooler->offset = 0;
+
+        return 0;
+    }
+
+    return -1;
+}
+
 int Unified2ReadEvent6Record(void *sph)
 {
     ssize_t                bytes_read;
@@ -265,6 +319,50 @@
     return -1;
 }

+int Unified2ReadEvent6RecordV2(void *sph)
+{
+    ssize_t                bytes_read;
+    int                 record_size;
+    Spooler                *spooler = (Spooler *)sph;
+
+    record_size = sizeof(Unified2Event6_v2);
+
+    if(!spooler->record.data)
+    {
+        /* SnortAlloc will FatalError if memory can't be assigned */
+        spooler->record.data=SnortAlloc(record_size);
+    }
+
+    if (spooler->offset < record_size)
+    {
+        /* in case we don't have it already */
+        bytes_read = read(spooler->fd, spooler->record.data + spooler->offset,
+                    record_size - spooler->offset);
+
+        if(bytes_read == -1)
+        {
+            LogMessage("ERROR: read error: %s\n", strerror(errno));
+            return BARNYARD2_FILE_ERROR;
+        }
+
+        if(bytes_read + spooler->offset != record_size)
+        {
+            spooler->offset += bytes_read;
+            return BARNYARD2_READ_PARTIAL;
+        }
+
+#ifdef DEBUG
+        Unified2PrintEvent6RecordV2((Unified2Event6_v2 *)spooler->record.data);
+#endif
+
+        spooler->offset = 0;
+
+        return 0;
+    }
+
+    return -1;
+}
+
 int Unified2ReadPacketRecord(void *sph)
 {
     ssize_t                bytes_read;
@@ -366,10 +464,66 @@
         "  packet_action      = %d\n", evt->packet_action););
 }

+void Unified2PrintEventRecordV2(Unified2Event_v2 *evt)
+{
+    char                sip4[INET_ADDRSTRLEN];
+    char                dip4[INET_ADDRSTRLEN];
+
+    if(evt == NULL)
+        return;
+
+    Unified2PrintEventCommonRecord((Unified2EventCommon *)evt);
+
+    inet_ntop(AF_INET, &(evt->ip_source), sip4, INET_ADDRSTRLEN);
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  ip_source          = %s\n", sip4););
+
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  sport_itype        = %d\n", ntohs(evt->sport_itype)););
+    inet_ntop(AF_INET, &(evt->ip_destination), dip4, INET_ADDRSTRLEN);
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  ip_destination     = %s\n", dip4););
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  dport_icode        = %d\n", ntohs(evt->dport_icode)););
+
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  ip_protocol        = %d\n", evt->protocol););
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  packet_action      = %d\n", evt->packet_action););
+}
+
 void Unified2PrintEvent6Record(Unified2Event6 *evt)
 {
     char                sip6[INET6_ADDRSTRLEN];
     char                dip6[INET6_ADDRSTRLEN];
+
+    if(evt == NULL)
+        return;
+
+    Unified2PrintEventCommonRecord((Unified2EventCommon *)evt);
+
+    inet_ntop(AF_INET6, &(evt->ip_source), sip6, INET6_ADDRSTRLEN);
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  ip_source          = %s\n", sip6););
+
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  sport_itype        = %d\n", ntohs(evt->sport_itype)););
+    inet_ntop(AF_INET6, &(evt->ip_destination), dip6, INET6_ADDRSTRLEN);
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  ip_destination     = %s\n", dip6););
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  dport_icode        = %d\n", ntohs(evt->dport_icode)););
+
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  ip_protocol        = %d\n", evt->protocol););
+    DEBUG_WRAP(DebugMessage(DEBUG_LOG,
+        "  packet_action      = %d\n", evt->packet_action););
+}
+
+void Unified2PrintEvent6RecordV2(Unified2Event6_v2 *evt)
+{
+    char                sip6[INET6_ADDRSTRLEN];
+    char                dip6[INET6_ADDRSTRLEN];

     if(evt == NULL)
         return;

The patch can also be downloaded here.

The code could probably be optimized since there are many duplicate lines now, but I did not want to change too much.
I sent the patch to the Barnyard2 developers and they may commit it in their git repository.
It looks like this patch should work on the 1.9-beta version as well, but I have not tested this… YMMV!

I have been running the patched Barnyard2 for several hours now on my Snort server without problems, using the vlan_event_types parameter. :)

Bookmark and Share

This entry was posted on Monday, October 11th, 2010 at 0:21 and is filed under Barnyard2, Patches. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

10 Responses to “Barnyard2 – Solving the “Unknown record type” errors”

  1. firdaus Says:

    im still a junior..where to paste the patch file

  2. Niels Horn Says:

    Hi,

    To use this patch file, you will need to download the sources of Barnyard2, apply the patch, compile the sources and create a new package for your Linux distribution.
    If this all sounds too complicated, you will need to find a friendly person that can do this tasks for you.
    I made packages for Slackware, the version of Linux I use personally, but each Linux “flavor” has its own way to create and install packages.

    I hope I at least helped you a little bit!

    Niels

  3. Dion Says:

    were you getting this error as well? FATAL ERROR: Unknown record type read: 110

  4. Niels Horn Says:

    Hi, Dion,

    No, I personally have not run into it yet, but it is caused by a new record type in Snort 2.9.0.
    It is called “Unified2_Extra_Data” and Barnyard2 needs to be updated to recognize it.

    I think you should bug the Barnyard2 developers for this and ask them to implement it :)
    For now, the solution is using the previous version of Snort (2.8.x), that does not use this new record type.

    Unfortunately I am without spare time at the moment to make a new patch, but it should not be too complicated…

    Do you have any idea what kind of package triggered this event? I have 2.9.0 running on my server but have not encountered it yet.

    Thanks for your feedback!

    Niels

  5. Dion Says:

    Niels,
    Thanks for the quick reply. I did send a email off to them to make them aware, but have not got a reply back yet. I think I will just go back to the older version to get this up and running. Thanks again for taking the time to reply.

  6. Niels Horn Says:

    No problem :)

    If you are a Barnyard2 user (or pretend to become one), it might be interesting to subscribe to the mailing list as well.

  7. Dion Says:

    Well I found out what was causing my issue. It was the enable_xff and inspect_gzip in the http_inspect area of snort.conf
    I commented those out and so far it is working. Thanks again

  8. Chito Says:

    After patching barnyard2, I did, make uninstall, make clean, ./configure –with-mysql && make && make install but no luck!! still getting 104 error..

  9. Chito Says:

    Oh, after deleting all logs from /var/log/snort/snort*.log the barnyard2 stop complaining

    Thanks for the patch!

  10. Niels Horn Says:

    Good that it worked out fine in the end!

    Niels

Leave a Reply



XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

It may take some time for your comment to appear, it is not necessary to submit it again.