class Cascading::Tap

A Tap represents a non-aggregate tap with a scheme, path, and optional sink_mode. c.t.l.FileTap is used in Cascading local mode and c.t.h.Hfs is used in Hadoop mode. Whether or not these can be created is governed by the :scheme parameter, which must contain at least one of :local_scheme or :hadoop_scheme. Schemes like TextLine are supported in both modes (by Cascading), but SequenceFile is only supported in Hadoop mode.

Attributes

path[R]
scheme[R]
sink_mode[R]

Public Class Methods

new(path, options = {}) click to toggle source

Builds a Tap given a required path

The named options are:

scheme

A Hash which must contain at least one of :local_scheme or :hadoop_scheme but may contain both. Default is text_line_scheme, which works in both modes.

#sink_mode

A symbol or string that may be :keep, :replace, or :append, and corresponds to the c.t.SinkMode enumeration. The default is :keep, which matches Cascading’s default.

# File lib/cascading/tap.rb, line 50
def initialize(path, options = {})
  @path = path

  @scheme = options[:scheme] || text_line_scheme
  raise "Scheme must provide one of :local_scheme or :hadoop_scheme; received: '#{scheme.inspect}'" unless scheme[:local_scheme] || scheme[:hadoop_scheme]

  @sink_mode = case options[:sink_mode] || :keep
    when :keep, 'keep'       then Java::CascadingTap::SinkMode::KEEP
    when :replace, 'replace' then Java::CascadingTap::SinkMode::REPLACE
    when :append, 'append'   then Java::CascadingTap::SinkMode::APPEND
    else raise "Unrecognized sink mode '#{options[:sink_mode]}'"
  end

  local_scheme = scheme[:local_scheme]
  @local_tap = local_scheme ? Java::CascadingTapLocal::FileTap.new(local_scheme, path, sink_mode) : nil

  hadoop_scheme = scheme[:hadoop_scheme]
  @hadoop_tap = hadoop_scheme ? Java::CascadingTapHadoop::Hfs.new(hadoop_scheme, path, sink_mode) : nil
end