class Cascading::Mode

A Mode encapsulates the idea of the execution mode for your flows. The default is Hadoop mode, but you can request that your code run in Cascading local mode. If you subsequently use a tap or a scheme that has no local implementation, the mode will be converted back to Hadoop mode.

Attributes

local[R]

Public Instance Methods

connect_flow(properties, name, sources, sinks, pipes) click to toggle source

Builds a c.f.Flow given properties, name, sources, sinks, and pipes from a Flow. The current mode is adjusted based on the taps and schemes of the sources and sinks, then the correct taps are selected before building the flow.

# File lib/cascading/mode.rb, line 44
def connect_flow(properties, name, sources, sinks, pipes)
  update_local_mode(sources, sinks)
  sources = select_taps(sources)
  sinks = select_taps(sinks)

  # Report execution mode to stdout before connecting
  puts "Connecting flow '#{name}' in #{local ? 'Cascading local mode' : 'Hadoop mode'}"

  flow_connector_class.new(java.util.HashMap.new(properties)).connect(name, sources, sinks, pipes)
end

source_tap(name, tap) click to toggle source

Attempts to select the appropriate tap given the current mode. If that tap does not exist, it fails over to the other tap with a warning.

# File lib/cascading/mode.rb, line 29
def source_tap(name, tap)
  warn "WARNING: No local tap for source '#{name}' in tap #{tap}" if local && !tap.local?
  warn "WARNING: No Hadoop tap for source '#{name}' in tap #{tap}" if !local && !tap.hadoop?

  if local
    tap.local_tap || tap.hadoop_tap
  else
    tap.hadoop_tap || tap.local_tap
  end
end

Public Class Methods

new(local) click to toggle source

Constructs a Mode given a flag indicating if it should be Cascading local mode.

# File lib/cascading/mode.rb, line 23
def initialize(local)
  @local = local
end

parse(mode) click to toggle source

Parses a specification of which mode, Cascading local mode or Hadoop mode, to execute in. Defaults to Hadoop mode. You may explicitly request Cascading local mode with values ‘local’ or :local. If you pass a Mode object to this method, it will be passed through.

# File lib/cascading/mode.rb, line 13
def self.parse(mode)
  case mode
  when Mode then mode
  when 'local', :local then Mode.new(true)
  else Mode.new(false)
  end
end