< first, install fex; assume you have downloaded class library version from ccg web page> mssammon|csil-mm24|/scratch/tutorial/preprocessing|[10]% cd fex mssammon|csil-mm24|/scratch/tutorial/preprocessing/fex|[11]% ls fexClient.pl known_name.lst org_words.lst Sensor.h Fex.cpp known_org.lst Parser.cpp Sentence.h FexGlobalParams.h known_place.lst Parser.h Sparser.h Fex.h known_state.lst README2 Sparser.y FexParams.cpp known_title.lst Revision.txt Statcalc.cpp FexParams.h language.lst* RGF.cpp Stats.cpp Flex.c Lexicon.cpp RGF.h Stats.h Flex.h Lexicon.h sample_script* temporal_words.lst* Flex.l Main.cpp Scanner.cpp tokens.h known_city.lst Makefile Scanner.h tools/ known_country.lst nationality.lst* Sensor.cpp verbClass.txt mssammon|csil-mm24|/scratch/tutorial/preprocessing/fex|[12]% gmake /usr/bin/g++ -O2 -c -DLINUX -o Fex.o Fex.cpp /usr/bin/g++ -O2 -c -DLINUX -o FexParams.o FexParams.cpp /usr/bin/g++ -O2 -c -DLINUX -o Lexicon.o Lexicon.cpp /usr/bin/g++ -O2 -c -DLINUX -o Parser.o Parser.cpp /usr/bin/g++ -O2 -c -DLINUX -o Sparser.o Sparser.cpp /usr/bin/g++ -O2 -c -DLINUX -o Flex.o Flex.c /usr/bin/g++ -O2 -c -DLINUX -o RGF.o RGF.cpp /usr/bin/g++ -O2 -c -DLINUX -o Sensor.o Sensor.cpp ar rc libfex.a Fex.o FexParams.o Lexicon.o Parser.o Sparser.o Flex.o RGF.o Sensor.o ranlib libfex.a /usr/bin/g++ -O2 -c -DLINUX -o Main.o Main.cpp /usr/bin/g++ -O2 -o fex Main.o -lfl -pthread -L/scratch/tutorial -lfex /usr/bin/ld: cannot find -lfex collect2: ld returned 1 exit status gmake: *** [fex] Error 1 mssammon|csil-mm24|/scratch/tutorial/preprocessing/fex|[14]% xemacs Makefile mssammon|csil-mm24|/scratch/tutorial/preprocessing/fex|[15]% gmake /usr/bin/g++ -O2 -o fex Main.o -lfl -pthread -L. -lfex mssammon|csil-mm24|/scratch/tutorial/preprocessing/fex|[16]% ls < here I'm assuming you've downloaded the tarball spell_prep.tar.gz from the tutorial page's link "preprocessing test materials" > mssammon|csil-mm24|/scratch/tutorial/preprocessing|[1]% ls fex/ pos-tagger/ running_prep.script spell_prep.tar.gz HONORIFICS README sentence-boundary.pl* word-splitter.pl mssammon|csil-mm24|/scratch/tutorial/preprocessing|[2]% tar xzf spell_prep.tar.gz mssammon|csil-mm24|/scratch/tutorial/preprocessing|[3]% ls fex/ README spell_prep/ HONORIFICS running_prep.script spell_prep.tar.gz pos-tagger/ sentence-boundary.pl* word-splitter.pl mssammon|csil-mm24|/scratch/tutorial/preprocessing|[4]% cd spell_prep mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[5]% ls accept-except.scr accept-except.targ accept-except-simple.scr generate_examples.sh* mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[8]%cd .. mssammon|csil-mm24|/scratch/tutorial/preprocessing|[9]% mv *gz ../tars mssammon|csil-mm24|/scratch/tutorial/preprocessing|[18]% perl sentence-boundary.pl Usage: boundary.pl -d honorifics_file -i input_file -o output_file mssammon|csil-mm24|/scratch/tutorial/preprocessing|[19]% perl sentence-boundary.pl -d HONORIFICS -i short_ae.txt -o short_ae.sent mssammon|csil-mm24|/scratch/tutorial/preprocessing|[20]% cat short_ae.sent But Sheik Ali of Kuwait won't accept any increase in the overall ceiling. Directors said all operating divisions, except the agribusiness group, improved their performances. mssammon|csil-mm24|/scratch/tutorial/preprocessing|[21]% ./word-splitter.pl usage: word-splitter.pl [ispell] mssammon|csil-mm24|/scratch/tutorial/preprocessing|[22]% ./word-splitter.pl ae.sent But Sheik Ali of Kuwait won 't accept any increase in the overall ceiling . Directors said all operating divisions , except the agribusiness group , improved their performances . ". > mssammon|csil-mm24|/scratch/tutorial/preprocessing|[23]% ./word-splitter.pl short_ae.sent > short_ae.word mssammon|csil-mm24|/scratch/tutorial/preprocessing|[24]% cat short_ae.word But Sheik Ali of Kuwait won 't accept any increase in the overall ceiling . Directors said all operating divisions , except the agribusiness group , improved their performances . mssammon|csil-mm24|/scratch/tutorial/preprocessing|[25]% ls fex/ README sentence-boundary.pl~* short_ae.word HONORIFICS running_prep.script short_ae.sent spell_prep/ pos-tagger/ sentence-boundary.pl* short_ae.txt word-splitter.pl* mssammon|csil-mm24|/scratch/tutorial/preprocessing|[26]% cp short_ae.word pos-tagger mssammon|csil-mm24|/scratch/tutorial/preprocessing|[27]% cd pos-tagger mssammon|csil-mm24|/scratch/tutorial/preprocessing/pos-tagger|[28]% ls compare-labels.cc data/ generate.cc net/ tagger.cc compile.sh* features.cc generate.h README.POS tagger.o corpus.test features.h generate.o SendReceive.h corpus.test.tagged features.o lex-examples.cc short_ae.word create-lexicon* gen* Makefile snow/ create-lexicon.cc gen.cc Makefile~ tagger* mssammon|csil-mm24|/scratch/tutorial/preprocessing/pos-tagger|[29]% ./tagger Usage: ./tagger -i : corpus input file -o : corpus output file -s : server port number mssammon|csil-mm24|/scratch/tutorial/preprocessing/pos-tagger|[30]% ./tagger -i short_ae.word -o short_ae.pos mssammon|csil-mm24|/scratch/tutorial/preprocessing/pos-tagger|[31]% cat short_ae.pos (CC But) (NNP Sheik) (NNP Ali) (IN of) (NNP Kuwait) (VBD won) (NN 't) (VB accept) (DT any) (NN increase) (IN in) (DT the) (JJ overall) (NN ceiling) (. .) (NNS Directors) (VBD said) (DT all) (VBG operating) (NNS divisions) (, ,) (IN except) (DT the) (NN agribusiness) (NN group) (, ,) (VBD improved) (PP$ their) (NNS performances) (. .) mssammon|csil-mm24|/scratch/tutorial/preprocessing/pos-tagger|[32]% cd .. mssammon|csil-mm24|/scratch/tutorial/preprocessing|[33]% ls fex/ README sentence-boundary.pl~* short_ae.word HONORIFICS running_prep.script short_ae.sent spell_prep/ pos-tagger/ sentence-boundary.pl* short_ae.txt word-splitter.pl mssammon|csil-mm24|/scratch/tutorial/preprocessing|[38]% cd spell_prep mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[39]% ln -s ../fex/fex mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[40]% ls accept-except.scr accept-except.targ generate_examples.sh* accept-except-simple.scr fex@ short_ae.txt mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[41]% ./fex Fex - Feature Extractor Cognitive Computations Group - University of Illinois at Urbana/Champaign Version 2.3.1 Usage: fex [options] mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[42]% mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[43]%cp ../pos-tagger/short_ae.pos . mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[44]% cat generate_examples.sh #!/bin/sh rm -f accept-except*.ex ./fex -t accept-except.targ accept-except-simple.scr accept-except-simple.lex short_ae.pos accept-except-simple.ex ./fex -t accept-except.targ accept-except.scr accept-except.lex short_ae.pos accept-except.ex mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[45]% ./generate_examples.sh Fex - Feature Extractor Cognitive Computations Group - University of Illinois at Urbana/Champaign Version 2.3.1 Processing... Fex - Feature Extractor Cognitive Computations Group - University of Illinois at Urbana/Champaign Version 2.3.1 Processing... mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[46]% ls accept-except.ex accept-except-simple.lex generate_examples.sh* accept-except.lex accept-except-simple.scr short_ae.pos accept-except.scr accept-except.targ accept-except-simple.ex fex@ mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[47]% cat accept-except-simple.lex ... mssammon|csil-mm24|/scratch/tutorial/preprocessing/spell_prep|[47]% cat accept-except-simple.ex ...