This project is not maintained anymore! It has been marked as dormant by Apache Logging Services consensus on 2024-10-10. Users are advised to migrate to alternatives. For other inquiries, see the support policy.
™ Apache Flume™ Apache Software Foundation

Version 1.5.0

Status of this release

Apache Flume 1.5.0 is the fifth release of Flume as an Apache top-level project (TLP). Apache Flume 1.5.0 is production-ready software.

Release Documentation

Changes

Release Notes - Flume - Version v1.5.0

** New Feature
  • [FLUME-1227] - Introduce some sort of SpillableChannel
  • [FLUME-2056] - Allow SpoolDir to pass just the filename that is the source of an event
  • [FLUME-2071] - Flume Context doesn’t support float or double configuration values.
  • [FLUME-2185] - Upgrade morphlines to 0.7.0
  • [FLUME-2188] - flume-ng-log4jappender Support user supplied headers
  • [FLUME-2225] - Elasticsearch Sink for ES HTTP API
  • [FLUME-2294] - Add a sink for Kite Datasets
  • [FLUME-2309] - Spooling directory should not always consume the oldest file first.
** Improvement
  • [FLUME-1542] - Flume User Guide sample configurations need fixing
  • [FLUME-1899] - Make SpoolDir work with Sub-Directories
  • [FLUME-2052] - Spooling directory source should be able to replace or ignore malformed characters
  • [FLUME-2130] - Handle larger payloads via SyslogUDPSource
  • [FLUME-2139] - upgrade morphline library dependency to a 0.5.0 release
  • [FLUME-2154] - Reducing duplicate events caused by reset-connection-interval
  • [FLUME-2155] - Improve replay time
  • [FLUME-2181] - Optionally disable File Channel fsyncs
  • [FLUME-2202] - AsyncHBaseSink should coalesce increments to reduce RPC roundtrips
  • [FLUME-2206] - ElasticSearchSink ttl field modification to mimic Elasticsearch way of specifying TTL
  • [FLUME-2207] - HDFS file suffix style index suffix in ElasticSearchSink
  • [FLUME-2212] - upgrade to Morphlines-0.8.0
  • [FLUME-2213] - MorphlineInterceptor should share metric registry across threads for better (aggregate) reporting
  • [FLUME-2217] - Preserve priority, timestamp and hostname fields in MultiportSyslogTcp and Udp sources
  • [FLUME-2231] - Add details in Flume Ganglia config in User Guide
  • [FLUME-2243] - AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory
  • [FLUME-2267] - Increase default transactionCapacity for FileChannel to 10000 from 1000
  • [FLUME-2275] - Improve scalability of MorphlineInterceptor under contention
  • [FLUME-2292] - Upgrade mapdb to 0.9.8
  • [FLUME-2316] - Upgrade MorphlineSolrSink to kite-0.12.0
  • [FLUME-2340] - Refactor to make room for Morphlines Elasticsearch Sink
  • [FLUME-2343] - Add user impersonation to DatasetSink
  • [FLUME-2351] - Ability to override any parameter from the configuration file
  • [FLUME-2352] - HDFSCompressedDataStream should support appendBatch
** Bug
  • [FLUME-1666] - Syslog source strips timestamp and hostname from log message body
  • [FLUME-1679] - Add dependency on Guava to flume-ng-elasticsearch-sink POM
  • [FLUME-1892] - IRC Sink NPE
  • [FLUME-1951] - Remove unused future from FlumeEventQueue
  • [FLUME-2007] - HDFS Sink should check if file is closed and retry if it is not.
  • [FLUME-2088] - Minor typo in Flume User Guide JSON Reporting section
  • [FLUME-2109] - HTTPS support in HTTP Source
  • [FLUME-2110] - Scribe Source must check if category is null before inserting the headers
  • [FLUME-2119] - duplicate files cause flume to enter irrecoverable state
  • [FLUME-2121] - Upgrade Flume to log4j 1.2.17 to be in sync with HDFS/HBase
  • [FLUME-2122] - Minor cleanups of User guide
  • [FLUME-2123] - Morphline Solr sink missing short type name
  • [FLUME-2124] - Upgrade Morphline Solr Sink to CDK 0.4.1
  • [FLUME-2127] - JMX shutdown command for Flume
  • [FLUME-2134] - AsyncHbase Sink bugfix plus tests errors on Windows
  • [FLUME-2135] - Add zip to the build distribution for Windows support
  • [FLUME-2136] - Windows - Fix intermitent test failure in TestMonitoredCounterGroup.java
  • [FLUME-2137] - Fix StagedInstall.java to invoke the correct startup script on Windows
  • [FLUME-2142] - HTTPS tests for http source
  • [FLUME-2145] - TestCheckpointRebuilder.testFastReplay fails on Windows due to checkpoint file being memory mapped
  • [FLUME-2151] - Windows: Update TestExecSource to use native commands on Windows
  • [FLUME-2152] - Flume user guide says “Ganglia support” where it should say “JSON support”
  • [FLUME-2156] - Unregister then re-register MonitoredCounterGroup JMX MBeans on reconfigure
  • [FLUME-2157] - Spool directory source does not shut down correctly when Flume is reconfigured
  • [FLUME-2159] - Sporadic failures in TestNettyAvroRpcClient.spinThreadsCrazily()
  • [FLUME-2161] - Flume does not support spaces in -X java-opt command line args
  • [FLUME-2172] - Update protocol buffer from 2.4.1 to 2.5.0
  • [FLUME-2176] - SpoolDir Source, get ‘File has changed’ exception but actually there is no change on the file
  • [FLUME-2182] - Spooling Directory Source will not ingest data completely when a wide character appears at the edge of a buffer
  • [FLUME-2184] - flume-ng-morphline-solr-sink Build failing due to incorrect hadoop-common dependency declaration
  • [FLUME-2191] - HDFS Minicluster tests failing after protobuf upgrade.
  • [FLUME-2192] - AbstractSinkProcessor stop incorrectly calls start
  • [FLUME-2198] - Avro Source should disable itself if ipFilterRules contains invalid rules
  • [FLUME-2199] - Flume builds with new version require mvn install before site can be generated
  • [FLUME-2200] - HTTP Source should be able to use “port” parameter if SSL is enabled
  • [FLUME-2208] - Jetty’s default SocketSelector leaks File descriptors
  • [FLUME-2209] - AsyncHBaseSink will never recover if the column family does not exists for the first start
  • [FLUME-2210] - UnresolvedAddressException when using multiple hostNames in Elasticsearch sink configuration
  • [FLUME-2220] - ElasticSearch sink - duplicate fields in indexed document
  • [FLUME-2229] - Backoff period gets reset too often in OrderSelector
  • [FLUME-2233] - MemoryChannel lock contention on every put due to bytesRemaining Semaphore
  • [FLUME-2235] - idleFuture should be cancelled at the start of append
  • [FLUME-2238] - Provide option to configure worker threads in NettyAvroRpcClient
  • [FLUME-2239] - Clarify File Channel’s dataDirs setting in User Guide
  • [FLUME-2252] - Add null check before closing table in HbaseSink
  • [FLUME-2253] - Please delete old releases from mirroring system
  • [FLUME-2255] - Spooling Directory Source cannot handle channel exceptions
  • [FLUME-2259] - transaction closure not happening for all the scenario in hbasesink
  • [FLUME-2262] - Log4j Appender should use timeStamp field not getTimestamp
  • [FLUME-2263] - Bump Hadoop 2 version to 2.3.0
  • [FLUME-2264] - Log4j Appender + Avro Reflection on string results in an invalid avro schema
  • [FLUME-2265] - Closed bucket writers should be removed from sfwriters map
  • [FLUME-2266] - Update Morphline Sink to kite-0.10.0
  • [FLUME-2270] - Twitter Source Documentation Does not load properly
  • [FLUME-2272] - Getting start page returning 503 error
  • [FLUME-2283] - Spool Dir source must check interrupt flag before writing to channel
  • [FLUME-2289] - Disable maxUnderReplication test which is extremely flakey
  • [FLUME-2301] - Update HBaseSink tests to reflect sink returning backoff only on empty batches
  • [FLUME-2302] - TestHDFS Sink fails with Can’t get Kerberos realm
  • [FLUME-2303] - HBaseSink tests can fail based on order of execution
  • [FLUME-2304] - DatasetSink test fails unexpectedly
  • [FLUME-2305] - BucketWriter#close must cancel idleFuture
  • [FLUME-2307] - Remove Log writetimeout
  • [FLUME-2311] - Use standard way of finding queue/topic
  • [FLUME-2312] - Add utility for adorning HTTP contexts in Jetty
  • [FLUME-2314] - Upgrade to Mapdb 0.9.9
  • [FLUME-2320] - Deadlock in DatasetSink
  • [FLUME-2323] - Morphline sink must increment eventDrainAttemptCount when it takes event from channel
  • [FLUME-2324] - Support writing to multiple HBase clusters using HBaseSink
  • [FLUME-2325] - BucketWriter might throw BucketClosedException incorrectly
  • [FLUME-2328] - FileChannel Dual Checkpoint Backup Thread not released on Application stop
  • [FLUME-2329] - Add an alias for the Morphline Solr Sink
  • [FLUME-2330] - Remove the MorphlineHandlerImpl configuration option from MorphlineSink
  • [FLUME-2334] - Upgrade Asynchbase to 1.5.0 as this include hbase 0.96.x support
  • [FLUME-2335] - TestHBaseSink#testWithoutConfigurationObject() must delete the table at the end of the test
  • [FLUME-2336] - HBase tests that pass in ZK configs must use a new context object
  • [FLUME-2338] - Support coalescing increments in HBaseSink
  • [FLUME-2345] - Update to Kite 0.12.0 dependency
  • [FLUME-2347] - Add FLUME_JAVA_OPTS which allows users to inject java properties from cmd line
  • [FLUME-2350] - Consume Order tests need to space out file creation
  • [FLUME-2357] - HDFS sink should retry closing files that previously had close errors
  • [FLUME-2381] - Upgrade Hadoop version in Hadoop 2 profile to 2.4.0
  • [FLUME-2379] - Flume’s pom files are invalid and not parseable by non-maven build tools
** Documentation
  • [FLUME-1223] - Userguide improvement rolling file sink
  • [FLUME-1678] - Incorrect documentation for HBase sink
  • [FLUME-1851] - User Guide grammar mistake
  • [FLUME-2064] - Typo/Grammar in flume main user doc under Scribe
  • [FLUME-2065] - Regex Extractor Interceptor config agent name inconsistent with rest of docs
  • [FLUME-2183] - Add “Other Resources” page to wiki
  • [FLUME-2278] - Incorrect documentation for write-timeout of File Channel
  • [FLUME-2319] - Incorrect property name of Ganglia Reporting documentation

Apache Flume

Resources

Apache

© Copyright 2009-2023 The Apache Software Foundation. Apache Flume, Flume, Apache, the Apache feather logo, and the Apache Flume project logo are trademarks of The Apache Software Foundation..