Logstash

Latest version: v0.1dev

Safety actively analyzes 714875 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 6 of 9

1.1.6

Overview of this release:
- new inputs: drupal_dblog.
- new filters: anonymize, metrics.
- new outputs: syslog, cloudwatch.
- new 'charset' setting for all inputs. This should resolve all known encoding
problems. The default charset is UTF-8.
- grok now captures (?<somename>...) regexp into 'somename' field
- Elasticsearch 0.20.2 is included. This means you are required to upgrade
your elasticsearch cluster to 0.20.2. If you wish to continue using an old
version of elasticsearch, you should use the elasticsearch_http plugin
instead of the elasticsearch one.

general
- fixed internal dependency versioning on 'addressable' gem (LOGSTASH-694)
- fixed another case of 'watchdog timeout' (LOGSTASH-701)
- plugin flags are now deprecated. The grok filter (--grok-pattern-path) was
the only plugin to make use of this.
- the grok filter has improved documentation
- lots of documentation fixes (James Turnbull, Louis Zuckerman)
- lots of testing improvements (Philippe Weber, Laust Rud Jacobsen)
- all 'name' settings have been deprecated in favor of more descriptive
settings (LOGSTASH-755)
- JRuby upgraded to 1.7.1
- removed use of bundler
- Fixed timestamp parsing in MRI (patch by Rene Lengwinat)

inputs
- All inputs now have a 'charset' setting to help you inform logstash of the
text encoding of the input. This is useful if you have Shift_JIS or CP1251
encoded log files. This should help resolve the many UTF-8 bugs that were
reported recently. The default charset is UTF-8.
- new: drupal_dblog: read events from a DBLog-enabled Drupal. (251, Patch by
theduke)
- bugfix: zeromq: 'topology' is now a required setting
- bugfix: lumberjack: client connection closing is now handled properly.
(Patch by Nick Ethier)
- misc: lumberjack: jls-lumberjack gem updated to 0.0.7
- bugfix: stomp: fix startup problems causing early termination (226
- bugfix: tcp: the 'source host' for events is now the client ip:port that
sent it, instead of the listen address that received it. (LOGSTASH-796)
- improvement: tcp: the default data_timeout is now -1 (never timeout).
This change was made because read timeouts were causing data loss, and
logstash should avoid losing events by default.
- improvement: amqp: the 'name' setting is now called 'queue' (274)
- improvement: eventlog: the 'name' setting is now called 'logfile' (274)
- bugfix: log4j: fix stacktrace reading (253, patch by Alex Arutyunyants)

filters
- new: anonymize: supports many hash mechanisms (murmur3, sha1, md5, etc) as
well as IP address anonymization (280, 261; patches by Richard Pijnenburg
and Avishai Ish-Shalom)
- new: metrics: allows you to aggregate metrics from events and emit them
periodically. Think of this like 'statsd' but implemented as a logstash
filter instead of an external service.
- feature: date: now accepts 'match' as a setting. Use of this is preferable
to the old syntax. Where you previously had 'date { somefield =>
"somepattern" }' you should now do: 'date { match => [ "somefield",
"somepattern" ] }'. (248, LOGSTASH-734, Patch by Louis Zuckerman)
- feature: grok: now accepts (?<foo>...) named captures. This lets you
compose a pattern in the grok config without needing to define it in a
patterns file. Example: (?<hostport>%{HOST}:%{POSINT}) to capture 'hostport'
- improvement: grok: allow '$' in JAVACLASS pattern (241, patch by Corry
Haines)
- improvement: grok: can now match against number types. Example, if you're
sending a json format event with { "status": 403 } you can now grok that
field. The number is represented as a string "403" before pattern matching.
- bugfix: date: Fix a bug that would crash the pipeline if no date pattern
matched. (LOGSTASH-705)
- feature: kv: Adds field_split, value_split, prefix, and container
settings. (225, patch by Alex Wheeler)
- bugfix: mutate: rename on a nonexistent field now does nothing as expected.
(LOGSTASH-757)
- bugfix: grok: don't tag an event with _grokparsefailure if it's already so
(248, patch by Greg Brockman)
- feature: mutate: new settings - split, join, strip. "split" splits a field
into an array. "join" merges an array into a string. "strip" strips leading and
trailing whitespace. (Patch by Avishai Ish-Shalom)

outputs
- new: syslog output supporting both RFC3164 and RFC5424 (180, patch by
Rui Alves)
- new: cloudwatch output to emit metrics and other events to Amazon CloudWatch.
(LOGSTASH-461, patch by Louis Zuckerman)
- feature: stdout: added 'message' setting for configuring the output message
format. The default is same behavior as before this feature.
- feature: http: added 'format' option to select 'json' or form-encoded
request body to send with each request.
- feature: http: added 'content_Type' option set the Content-Type header.
This defaults to "application/json" if the 'format' is 'json'. Will default
to 'application/x-www-form-urlencoded' if the 'format' is 'form'
- bugfix: zeromq: 'topology' is now a required setting
- feature: mongodb: new setting 'isodate' that, when true, stores the
timestamp field as a mongodb date instead of a string. (224, patch by
Kevin Amorin)
- improvement: gelf: Allow full_message gelf property to be overridden (245,
patch by Sébastien Masset)
- misc: lumberjack: jls-lumberjack gem updated to 0.0.6
- feature: nagios: New 'nagios_level' setting to let you change the level
of the passive check result sent to nagios. (298, Patch by James Turnbull)
- feature: elasticsearch, elasticsearch_http, elasticsearch_river: new setting
'document_id' for explicitly setting the document id in each write to
elasticsearch. This is useful for overwriting existing documents.
- improvement: elasticsearch_river: 'name' is now 'queue' (274)
- improvement: amqp: 'name' is now 'exchange' (274)
- bugfix: the websocket output works again (supports RFC6455)

1.1.5

Overview of this release:
* New inputs: zenoss, gemfire
* New outputs: lumberjack, gemfire
* Many UTF-8 crashing bugs were resolved

general
- new runner command 'rspec' - lets you run rspec tests from the jar
This means you should now be able to write external tests that execute your
logstash configs and verify functionality.
- "file not found" errors related to paths that had "jar:" prefixes should
now work. (Fixes LOGSTASH-649, LOGSTASH-642, LOGSTASH-655)
- several plugins received UTF-8-related fixes (file, lumberjack, etc)
File bugs if you see any UTF-8 related crashes.
- 'json_event' format inputs will now respect 'tags' (239, patch by
Tim Laszlo)
- logstash no longer uses nor recommends bundler (see 'gembag.rb'). The
Gemfile will be purged in the near future.
- amqp plugins are now marked 'unsupported' as there is no active maintainer
nor is there source of active support in the community. If you're interested
in maintainership, please email the mailing list or contact Jordan!

inputs
- irc: now stores irc nick
- new: zenoss (232, patch by Chet Luther)
- new: gemfire (235, patch by Andrea Campi)
- bugfix: udp: skip close() call if we're already closed (238, patch by kcrayon)

filters
- bugfix: fix for zeromq filter initializer (237, patch by Tom Howe)

outputs
- new: lumberjack output (patch by Nick Ethier)
- new: gemfire output (234, patch by Andrea Campi)
- improved: nagios_ncsa (patch by Tomas Doran)
- improved: elasticsearch: permit setting 'host' even if embedded. Also set the
host default to 'localhost' when using embedded. These fixes should help resolve
issues new users have when their distros surprisingly block multicast by
default.
- improved: elasticsearch: failed index attempts will be retried
- improved: irc: new 'password' setting (283, patch by theduke)

1.1.4

Overview of this release:
- bug fixes mostly

filters
- date: Fix crashing on date filter failures. Wrote test to cover this case.
(LOGSTASH-641)
- grok: Improve QUOTEDSTRING pattern to avoid some more 'watchdog timeout' problems

outputs
- nagios_nsca: Allow check status to be set from the event (228, patch by
Tomas Doran)
- elasticsearch_http: Fix OpenSSL::X509::StoreError (LOGSTASH-642)

1.1.3

- rebuilt 1.1.2 for java 5 and 6

1.1.2

Overview of this release:
* New input plugins: lumberjack, sqs, relp
* New output plugins: exec, sqs
* New filter plugins: kv, geoip, urldecode, alter
* file input supports backfill via 'start_position'
* filter watchdog timer set to 10 seconds (was 2 seconds)

general
- Stopped using 'Gemfile' for dependencies, the logstash.gemspec has returned.
(Patch by Grant Rogers)
- New 'logstash-event.gemspec' for generating logstash events in your own
ruby programs (Patch by Garry Shutler)
- Wildcard config files are now sorted properly (agent -f
/etc/logstash/*.conf)
- The old '-vvv' setting ruby's internal $DEBUG is now gone. It was causing
too much confusion for users due to noise.
- Improved 'logstash event' creation speed by 3.5x
- Now uses JRuby 1.7.0
- Now ships with Elasticsearch 0.19.10

inputs
- bugfix: redis: [LOGSTASH-526] fix bug with password passing
- new: lumberjack: for use with the lumberjack log shipper
(https://github.com/jordansissel/lumberjack)
- new: sqs: Amazon SQS input (Patch by Sean Laurent, 211)
- new: relp: RELP (rsyslog) plugin (Patch by Mike Worth, 177)
- file input: sincedb path is now automatically generated if not specified.
This helps work around a problem where two file inputs don't specify a
sincedb_path would clobber eachother (LOGSTASH-554)
- file input: no longer crashes if HOME is not set in env (LOGSTASH-458)
- log4j input: now supports MDC 'event properties' which are stored as fields
in the logstash event. (216, 179. Patches by Charles Robertson and Jurjan
Woltman)
- pipe input: should work now.

filters
- new: kv: useful for parsing log formats taht use 'foo=bar baz=fizz' and
similar key-value-like things.
- new: urldecode: a filter for urldecoding fields in your event. (Patch by
Joey Imbasciano, LOGSTASH-612)
- new: geoip: query a local geoip database for location information (Patch by
Avishai Ish-Shalom, 208)
- improvement: zeromq: an empty reply is now considered as a 'cancel this
event' operation (LOGSTASH-574)
- bugfix: mutate: fix bug in uppercase and lowercase feature that would
prevent it from actually doing the uppercasing/lowercasing.
- improvement: mutate: do the 'remove' action last (LOGSTASH-543)
- feature: grok: new 'singles' config option which, when true, stores
single-value fields simply as a single value rather than as an array, like
[value]. (LOGSTASH-185)
- grok patterns: the URIPARAM pattern now includes pipe '|' as a valid
character. (Patch by Chris Mague)
- grok patterns: improve haproxy log patterns (Patch by Kevin Nuckolls)
- grok patterns: include 'FATAL' as a valid LOGLEVEL match
(patch by Corry Haines)
- grok patterns: 'ZONE' is no longer captured by name in the HTTPDATE pattern
- new: alter: adds some conditional field modification as well as a
'coalesce' feature which sets the value of a field to the first non-null
value given in a list. (Patch by Francesco Salbaroli)
- improvement: date: add TAI64N support
- improvement: date: subsecond precision on UNIX timestamps is retained on
conversion (213, Patch by Ralph Meijer)
- improvement: date: Add locale setting; useful for day/month name parsing.
(100, Patch by Christian Schröder)

outputs
- new: exec: run arbitrary commands based on an event.
- new: sqs: Amazon SQS output (Patch by Sean Laurent, 211)
- bugfix: redis: [LOGSTASH-526] fix bug with password passing
- improvement: redis: [LOGSTASH-573] retry on failure even in batch-mode. This
also fixes a prior bug where an exception in batch mode would cause logstash
to crash. (Patch by Alex Dean)
- improvement: riemann: metric and ttl values in riemann_event now support
sprintf %{foo} values. (pull 174)
- improvement: stdout: new 'dots' debug_format value emits one dot per event
useful for tracking event rates.
- gelf output: correct severity level mappings (patch by Jason Koppe)
- xmpp output: users and rooms are separate config settings now (patch by
Parker DeBardelaben)
- improvement: redis: 'host' setting now accepts a list of hosts for failover
of writes should the current host go down. (222, patch by Corry Haines)

1.1.1

Overview of this release:
* New input plugins: generator, heroku, pipe, ganglia, irc
* New output plugins: juggernaut, metricscatcher, nagios_ncsa, pipe,
opentsdb, pagerduty, irc
* New filter plugins: zeromq, environment, xml, csv, syslog_pri
* Fixes for gelf output
* Support for more than 1 filter worker (agent argument "-w")

Page 6 of 9

© 2025 Safety CLI Cybersecurity Inc. All Rights Reserved.