mediawikilexer.rb

The lexer for MediaWiki language.

Standalone usage:

 file = File.new("somefile", "r")
 input = file.read
 lexer = MediaWikiLexer.new
 lexer.tokenize(input)

Inside RACC-generated parser:

 ...
 ---- inner ----
 attr_accessor :lexer
 def parse(input)
     lexer.tokenize(input)
     return do_parse
 end
 def next_token
     return @lexer.lex
 end
 ...
 parser = MediaWikiParser.new
 parser.lexer = MediaWikiLexer.new
 parser.parse(input)