Open Text Summarizer

The open text summarizer is an open source command line tool for summarizing texts. The program reads a text and generates a summary. By default, the summarizer tries to reduce the text size down to 20%, for a this short text about python for instance, the summary would contain only the highlighted text.

Python is an interpreted, object-oriented, high-level programming language with dynamic semantics.
Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together.
Python’s simple, easy to learn syntax emphasizes readability and therefore reduces the cost of program maintenance.
Python supports modules and packages, which encourages program modularity and code reuse.
The Python interpreter and the extensive standard library are available in source or binary form without charge for all major platforms, and can be freely distributed. Often, programmers fall in love with Python because of the increased productivity it provides.
Since there is no compilation step, the edit-test-debug cycle is incredibly fast.
Debugging Python programs is easy: a bug or bad input will never cause a segmentation fault.
Instead, when the interpreter discovers an error, it raises an exception.
When the program doesn’t catch the exception, the interpreter prints a stack trace.
A source level debugger allows inspection of local and global variables, evaluation of arbitrary expressions, setting breakpoints, stepping through the code a line at a time, and so on.
The debugger is written in Python itself, testifying to Python’s introspective power.
On the other hand, often the quickest way to debug a program is to add a few print statements to the source: the fast edit-test-debug cycle makes this simple approach very effective.

It’s interface is pretty simple:

  ots [OPTION...] [file.txt | stdin]  - Open Text Summarizer
Help Options:
  -?, --help                Show help options
Application Options:
  -r, --ratio=<int>         summarization % [default = 20%]
  -d, --dic=<string>        dictionary to use
  -o, --out=<string>        output file [default = stdout]
  -h, --html                output as html
  -k, --keywords            only output keywords
  -a, --about               only output the summary
  -v, --version             show version information

Installation on Linux

Some Linux distributions come with OTS already pre-installed and otherwise the installation is very straight forward. For CentOS, the ots package can be found via rpmfind and deployed like so:


sudo yum install ./ots-libs-0.5.0-8.fc20.x86_64.rpm
sudo yum install ./ots-0.5.0-8.fc20.x86_64.rpm

While OTS version 5.0.x was installed, running the tool still reports an older version number.

ots -v
ots 0.4.2

Installation on OS X (including 10.9 Mavericks)

Installing OTS on a OS X is more involved, but can be done in about 10 to 15 minutes. The following steps require that XTools or XTools’s command-line tools are installed already.
1. Install Home Brew and some libs needed to build OTS

ruby -e "$(curl -fsSL"
GIT_SSL_NO_VERIFY=1 brew update
brew install wget 
brew install libxml2
brew install glib
brew install popt

2. Get the source:

mkdir ~/tmp
cd tmp
tar -xzvf ./ots-0.5.0.tar.gz 
cd ./ots-0.5.0

3. Configure will create the Makefile:


4. Edit the just created Makefile:remove doc at the end of this line: (around line 103)

SUBDIRS = src dic doc

5. Make and deploy:

sudo make install

6. Confirm

ots -v

Building on Cent OS 5.x Linux

As I’m sure you are very much accustomed to, building on CentOS 5.x is a pain in the ass. Here are some steps you may have to consider doing after uncompressing the source archive:

  • yum install glib2-devel
  • yum install libxml2-devel
  • ./configure
  • .. edit Makefile (see above)
  • make install
  • ln -s /usr/local/lib/libots* /usr/lib64

Accessing the OTS command line from a python program

While the open text summarizer is a useful tool to have, using it from within another program makes it so much more worthwhile.
Here for instance is how you can access OTS from python:

        content = content.encode('utf-8')
        content = str(content.decode('ascii', 'ignore'))
        content = " ".join(content.split()) # remove unnecessary white space

        temp_dir = tempfile.mkdtemp() # create temp directory and two temp files
        temp1 = tempfile.NamedTemporaryFile(suffix=".txt", dir = temp_dir, delete=False)
        temp2 = tempfile.NamedTemporaryFile(suffix=".txt", dir = temp_dir, delete=False)

        result = None
            #   write text content into file to be summarized
            r = 20
            cmdline = '{0} -r {1} -o {2} {3}'.format('/usr/local/bin/ots', str(20),,
            #   summarize into temp2
            result = subprocess.Popen(cmdline, shell=True, stdout=subprocess.PIPE).communicate()[0]            

            # cleanup
            return result


Leave a Reply