First there isn’t too much to get this ActiveX controller up in running in the first place. There are great examples here that take you 95% of the way towards getting these up and running. A few little tricks we learned when setting up the javascript portion of the code are as follows:

• To set the page size use the following method (the example sets it to letter):

        csxi.SetTwainLayout(0,8.5,0,11)

• Some scanners do not reset to their defaults after a custom scan. You need to make sure you manually set all default settings before scanning. A few you may want to set:

        csxi.UseTwainInterface=false; //this makes sure that you are not prompted to change settings
        csxi.TwainAutoBright = true;
        csxi.TwainAutoDeskew = true;

• When beginning a multipage scan you must run the following three methods:
  

        csxi.TwainMultiImage = true;
        csxi.UseADF = true;
        csxi.ClearPDF();

One other fun thing we can do with ChestySoft’s TwainX controller is upload the captured data directly to the server. To do this in the javascript you need to do the following:

      Success = csxi.PostImage(url, file_name, param_name, graphics_format);
      if (Success) {
        //what to do when success
      } else {
        //what to do if error - here are recommendations:
        csxi.SaveToFile(path_on_desktop) //write data to desktop if you saved as a graphic; 
        csxi.WritePDF(path_on_desktop) //write data to desktop if you saved as a pdf; 
      }

Some notes about the preceding code. url a fully qualified url (including the ‘http://’). file_name is what will be included as original_filename when you are reading data on the server side. graphics_format is a enumeration that can be found here.

And the best thing is after you click on the button that runs your scan the information uploaded is the same as a standard file upload so you don’t have to do anything strange at all in ruby.

In the project that I wrote this for we began by creating a wrapper class for the PDFTK gem mentioned earlier. The basic shell of the method we will be using accepts a list of files and a a title for the new pdf and begins and ends by creating and removing temp directories respectively.

class PdfTool< PDF::Toolkit
  def self.concat_and_make_table_of_contents(title, files)
    require 'pdf/writer'
    begin
      tmpdir = make_temp('')
      FileUtils.mkdir(tmpdir)
 
       #the rest of the code is going here   
 
    rescue => error
      raise error
    ensure
      FileUtils.rm(temppath)
    end
 
  end
end

Following this we are going to have set up PDF::Writer for creating the table of contents. Unfortunately PDF::Writer is a bit messy to get going, but we are only going to be creating a table of contents with the simplest format possible so it will not be too bad. Of course if you are not going to want this hard coded, we could easily rewrite the concat_and_make_table_of_contents so that it takes a hash of formatting variables. Here is the set up of PDF:Writer as well printing of the title.

      @text_font = "Times-Roman"
      @title_font_size = 17
      @title_vertical_spacing = 15
      @page_font_size = 12
      @page_vertical_spacing = 8
 
      print_pdf = PDF::Writer.new(:paper => "A4")
      print_pdf.margins_pt(36)
      @top_heading_font_position = 10
 
      print_pdf.select_font @text_font
      print_pdf.text("<b>#{title}</b>", :font_size => @title_font_size,:justification => :center,:left => @top_heading_font_position)
      print_pdf.move_pointer(@title_vertical_spacing)

Next we need to go through the files and generate the table of contents. We will have a loop to check that all of the files in the array exist and are PDFs (though this will need to change if you decide to add in html, etc conversion). Then we are going to start assuming the first page of the first document in the list is page 1 (if you want to include the table of contents in the page count you will mostly likely have to generate the table of contents once, then figure out how many pages is its then regenerate it with the offset – but we are simplifying this for this example). Following that we are going to loop though all of the files, determine the title from the metadata and print the title and the page number for each page. This will finish up by saving the table of contents to a temp file. (Note: don’t be confused we are keeping track of two pdf variables – pdf which corresponds to PDFTK instances and print_pdf which corresponds to PDF::Writer instances).

      files = files.delete_if {|f| !File.exists?(f) || File.mime_type?(f) != 'application/pdf'}
      current_page=1
      files.each do |f|
        pdf = open(f)
        page_title = ((pdf[:Title] || pdf[:File]) || File.basename(f))
        print_pdf.text("#{page_title} - #{current_page}", :font_size => @page_font_size,:justification => :left)
        print_pdf.move_pointer(@page_vertical_spacing)
        current_page+= pdf.pages
      end
 
      print_pdf.save_as(File.join(tmpdir,'toc.pdf'))

You may have noticed one method in the previous code that needs some flushing out. PDFTK provides accessors for all metadata that can be found in the pdf, you simply pass a symbol or a string to the [] method to retrieve it. Furthermore you can use PDFTK to access the number of pages in the file by simply calling pages.

For the last step we are going to concatenate all of the files that were passed in the array with the table of contents. Again PDFTK will come in handy here. There is a class method pdftk which will accept any commands that PDFTK will run as an array. So all we need to do is have the PDFTK invoke the concat method. Following that we are just going to read the data for the generated file and pass that back before we start rescuing and ensuring code. This portion is as follows:

      command = [File.join(tmpdir,'toc.pdf')] + files + ['cat','output',File.join(tmpdir, 'outfile.pdf')]
      raise IOError, "Unknown PDFTK Error command: pdftk #{command.join(' ')}" unless pdftk(*command)
 
      data = nil
      File.open(File.join(tmpdir, 'outfile.pdf'), 'r') { |f| data = f.read }
 
      return data

So now we have a simple method for concatenating and creating a table of contents for several PDF files.

  def object
    tmp = objects.first
    objects.each {|o| tmp.attributes.each {|key, value| tmp[key] = o[key] if value.blank? && key != 'id'}}
    tmp.freeze
  end

However there was a flaw here. Every time I would view the page all of the data in the objects was getting overwritten with one copy of it. After banging my head on the desk it was realized that tmp[key] = o[key] was actually writing the changes to the database permanently rewriting all of the objects (which still seems counter intuitive to me, because it seems like only the first record should have been the one changing). But the solution was pretty simple. The working method is as follows.

  def object
    tmp = Object.new
    objects.each {|o| tmp.attributes.each {|key, value| tmp[key] = o[key] if value.blank? && key != 'id'}}
    tmp.freeze
  end

The basic model looks as follows. make_temp is just a method that makes temp files/directories so you can set that up however you like.

class DocxTemplateTool < TemplateTool
  require 'find'
  require 'zip/zip'
  require 'zip/zipfilesystem'
 
  def initialize(data)
    @temp_path = make_temp(".zip")
    @temp_dir  = make_temp
    @data      = data
  end
end

Now we have can get started on reading the document and then replacing some text. We will start with the shell of method and then add stuff. In the beginning we need to save our data as a zip file (I should mention all of our data here is saved in blobs) and then unzip it’s contents in another directory. After we are done, we of course want to clean up all of these temp files. Both of the methods are going to be used by the replace method and by a validation method so to keep it DRY they are going into protected class methods.

  def replace(vars={})
    rdata = nil
    begin
      loadup
      #more code is going here
    ensure
      cleanup
    end
    return rdata
  end
 
  protected
  def loadup
      Dir.mkdir(@temp_dir, 777)
      File.open(@temp_path, 'wb') { |f| f.write data }
      args = ['unzip', @temp_path, '-d', @temp_dir]
      raise "Could not unzip #{@temp_path}" unless system(*args)
      args = ['chmod','-R','777', @temp_dir]
      raise "Could not chmod #{@temp_path}" unless system(*args)
  end
 
  def cleanup
      Dir.chdir(RAILS_ROOT)
      FileUtils.rm(@temp_path) if File.exists?(@temp_path)
      FileUtils.rm_rf(@temp_dir) if File.exists?(@temp_dir)
  end

The only piece here that might not make sense is the need to call Dir.chdir(RAILS_ROOT) in cleanup. The reason for this is that during the course of the replace method we may find ourselves in that directory and when we go to clean it up we won’t be able to delete it if we had cd‘ed to it.

Before we go onto the replace method there is one more protected method that we are going to have to write. This will method will check to make sure that as we are going through directories we are only doing replacement in the files we want to.

  protected
    def prunecheck?(path)
      FileTest.directory?(path) &&
      path != File.join(@temp_dir) &&
      path != File.join(@temp_dir,'word') &&
      path != File.join(@temp_dir,'docProps')
    end

As you look through the file structure you will see that we are ignoring all of the _rels directories since their purpose is to tell word which files do what and we do not want to ruin that. I also ignored the theme directory by my own choice, and we probably could choose to ignore other files n the word directory such as fontTable.xml, but as is this will allow you to replace anything in the document itself as well as in the metadata.

So finally we are going to do some replacement. Follows is the code that goes in #more code is going here.

      Find.find(File.join(@temp_dir)) do |path|
        Find.prune if (prunecheck?(path))
        unless FileTest.directory?(path)
          sdata = nil
          File.open(path, 'r') { |f| sdata = f.read }
          FileUtils.rm(path)
          File.open(path, 'w') { |f| f.write replace_data(sdata,vars) }
        end
      end
 
      FileUtils.rm(@temp_path) if File.exists?(@temp_path)
      Dir.chdir(@temp_dir)
      args = ['zip', '-r', @temp_path, '*']
      raise "Could not rezip file" unless system(args.join(" "))
      File.open(@temp_path, 'r') { |f| rdata = f.read }

We are beginning by using the Find.prune method to search through the directory structure for all files that we are going to be working on. If we find a file the process is as follows.

1. Read the data from the file into memory.
2. Remove the file.
3. Run a method on the data to do whatever replacements you would like and then save it back to where the file was before you deleted it.

I was not going to provide my replace_data method since it is particular to the project I wrote this for, but pretty much any sort of methods with gsub functions is what should be used. Lets just say replace_data looks like this.

  def replace_data(data, vars)
    data.gsub(/PHP/, 'Ruby')
  end

After the files have been replaced we are going to cd to the temp directory and rezip it. I chose to use a UNIX system call here since there were issues using Ruby’s zip class (if I recall correctly it was adding an extra folder when it was rezipping it). After that just, reread the zip file so that you can return that data after you are done cleaning up and voila you have a Word Document that doesn’t have any mention of PHP!

One final method we are going to add before we are done. It is very likely that you are going to want to make sure someone didn’t put anything malicious in the file that is going to mess with your system or maybe just let the user know why some text isn’t getting replaced. For that we have this method.

  def validate_data
    docx_errors = []
    begin
      loadup
      Find.find(File.join(@temp_dir)) do |path|
        Find.prune if (prunecheck?(path))
        File.open(path, 'r') { |f| docx_errors.concat(validate_template(f.read ))} unless FileTest.directory?(path)
      end
    ensure
      cleanup
    end
    return docx_errors
  end

It pretty much goes through the document just like we did with the replacement except we will call a method validate_template (that you will have to write) that will make sure your data is AOK!

After a few starts and stops, many hours of reading through the webdav documentation, and browsing through the http traffic using Fiddler – it was determined that locking was the issue.

What I found in the RailsDav plugin is that the implementation of locking is as follows (lines 92-100 of railsdav/act_as_railsdav.rb):

def webdav_lock()
    #TODO implementation for now return a 200 OK
    render :nothing => true, :status => 200 and return
end
 
def webdav_unlock()
    #TODO implementation for now return a 200 OK
    render :nothing => true, :status => 200 and return
end

However, Microsoft actually won’t let you edit a document with such lax locking. So I rewrote the lock method so that it returned a valid lock.

def webdav_lock()
     @lock = get_lock(@path_info)
     if @lock
         response.headers["Lock-Token"] = "<#{@lock.token}>"
         response.headers["Content-Type"] = 'text/xml; charset="utf-8"'
         render :inline => self.class.lock_xml, :layout => false, :type => :rxml, :status => 200  and return
       else
         render  :nothing => true, :status => WebDavErrors::ForbiddenError and return
       end
end

There were a few other pieces of code that were needed to get this working though. Namely the .lock_xml method, and get_lock. Let’s tackle them each in turn.

In railsdav/propxml.rb you can find the XML structures for the PROPFIND and the PROPPATCH methods for WebDAV. However both locking and unlocking require their own structures. So I added one for locking. It is as follows:

def lock_xml
<&lt;EOPROPFIND_XML 
         xml.D(:multistatus, {"xmlns:D" => "DAV:"}) do
           xml.D :lockdiscovery do
             xml.D :activelock do
               xml.D :locktype do 
                 xml.D @lock.type.to_sym
               end
               xml.D :lockscope do 
                 xml.D @lock.scope.to_sym
               end
               xml.D :depth, @lock.depth
               xml.D :timeout, @lock.timeout_full               
               xml.D :locktoken do 
                 xml.D :href, @lock.token
               end               
               xml.D :lockroot do 
                 xml.D :href, @lock.href
               end
             end
           end
         end  
EOPROPFIND_XML
end

Then there is the get_lock method. Writing get_lock was a two step process. If you look at (lines 269-287 of railsdav/act_as_railsdav.rb) you can see several methods that are very similar in function to their respective purposes. So I began by simply adding the following in this section:

def get_lock(path)
      raise WebDavErrors::ForbiddenError
end

Then in my application’s controller that overrides all of the acts_as_railsdav methods I did just that:

def get_lock(path)
    return false unless get_resource_for_path(path).locked?
    return ActiveRecordLock.new({:type => 'write', :scope => 'exclusive', :timeout => 60*60*5, :href => path, :id =>4})
end

So now you’ll notice two new things that need coding here. The .locked? method for the resource and ActiveRecordLock. We decided to not worry about multiple users editing the same document at the same time on this project. Because of this and the time constraints .locked? simply returns true. If one were to deal with unlocking at some point this would have to be come a check to the database or something to make sure that the document is indeed unlocked. The same goes for ActiveRecordLock. It really just deals with the flat information needed by the Lock XML and not saving it to check if it is locked later on. Maybe someday, somebody will undertake that task I suppose. Anyway, here is what I had for the portion of the ActiveRecordLock model that I needed.

class ActiveRecordLock
   attr_accessor :type, :scope, :timeout, :depth, :timeout_units, :href, :token
 
   def initialize(args)      
      @type    = args[:type]
      @scope   = args[:scope]
      @timeout = args[:timeout]
      @href    = args[:href]  
      @depth   = 'Infinity'
      @timeout_units = 'Second'
      @token   = build_token(args[:id])
   end
 
   def timeout_full
     "#{self.timeout_units}-#{self.timeout}"
   end
 
   protected
     def build_token(text)
       require 'digest/md5'
       md5 = Digest::MD5.hexdigest(text.to_s).to_s
       'opaquelocktoken:'+md5[0,7]+'-'+md5[8,11]+'-'+md5[12,15]+'-'+md5[16,19]+'-'+md5[20,31]
     end
end

There is still plenty that needs to be done to get the full lock/unlock, but hopefully this will get someone started and save a few headaches.