Rails 2.3 + Ruby 1.8 UTF-8 Hack

I work on a project which is currently still locked in Rails 2.3 running on Ruby 1.8 – of course, as years have gone by, more and more support for internationalization has come up, and now with emojis being part of the UTF-8 standard, and people people trying to use them in blog posts and comments and the like, I obviously encounter the fiasco that is trying to have Ruby on Rails on MySQL deal with this.

It’s been a mess.

In the end, I’ve just opted for a hack on the String class which gets used at the point that the model’s properties are assigned:

class String

  #
  # Converts multi-byte characters which use more than 2 bytes into HTML entities
  #
  def to_multibyte_html_entities
    each_char.map { |c| c.bytes.count > 2 ? "&#x#{c.multibyte_ord.to_s(16)};" : c }.join
  end
  
  #
  # Identical to #ord but properly supporting multibyte, like later versions
  # of Ruby
  #
  def multibyte_ord
    unpack('U')[0]
  end

end

Grails 3.3 with Servlet Filter

I had a scenario (I don’t really want to get into it) where I needed to implement a proper servlet filter which would run before the Grails dispatcher got involved in the request. The Grails documentation says it can be done, but the instructions are ridiculously vague, and seem more centered on plugins rather than just a webapp.

I finally managed to work it out, and I wanted to show the source code involved for posterity, and anyone else who might stumble upon this solution.

First off, create your filter in /src/main/java/myPackage:

package myPackage;

import java.io.IOException;
import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.annotation.WebFilter;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
 * @author Nathan Crause <[email protected]>
 */
@WebFilter(filterName = "testFilter", urlPatterns = {"/*"})
public class TestFilter implements Filter {

	private FilterConfig filterConfig;

	public TestFilter() {
	}

	/**
	 * @param request The servlet request we are processing
	 * @param response The servlet response we are creating
	 * @param chain The filter chain we are processing
	 *
	 * @exception IOException if an input/output error occurs
	 * @exception ServletException if a servlet error occurs
	 */
	@Override
	public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
			throws IOException, ServletException {
		HttpServletRequest httpRequest = (HttpServletRequest) request;
		HttpServletResponse httpResponse = (HttpServletResponse) response;

		for (int i = 0; i < 100; ++i) {
			for (int c = 0; c < 80; ++c) {
				System.out.print("-");
			}
			System.out.println();
		}

		chain.doFilter(request, response);
	}

	/**
	 * Destroy method for this filter
	 */
	@Override
	public void destroy() {		
	}

	/**
	 * Init method for this filter
	 * @param filterConfig
	 */
	@Override
	public void init(FilterConfig filterConfig) {		
		this.filterConfig = filterConfig;
	}

}

Now the really poorly documented part – where the hell do you tell Grails to load the damned thing? Turns out you just use /grails-app/conf/spring/resources.groovy:

import org.springframework.core.Ordered
import org.springframework.boot.web.servlet.FilterRegistrationBean
import myPackage.TestFilter

beans = {
	testFilter(FilterRegistrationBean) {
		filter = bean(TestFilter)
		urlPatterns = ['/*']
		// we want all other Grails filters to have loaded first
		order = Ordered.LOWEST_PRECEDENCE
	}
}

Now, I originally opted for using pure Java for the speed, but it did expose the problem that none of my domain classes were accessible, which I did require. So, I re-implemented the filter as a Groovy class in the /grails-app/controllers/myPackage directory.

It’s also worth noting that you will be required to create a new hibernate session in order to perform queries.

package myPackage

import javax.servlet.Filter
import javax.servlet.FilterChain
import javax.servlet.FilterConfig
import javax.servlet.ServletException
import javax.servlet.ServletRequest
import javax.servlet.ServletResponse
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse

/**
 * @author Nathan Crause <[email protected]>
 */
class TestFilter implements Filter {
	
	FilterConfig filterConfig
	
	void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) {
		for (int i = 0; i < 10; ++i) {
			println("-" * 80)
		}

		Account.withNewSession {
			println(Account.last().inspect())
		}

		chain.doFilter(request, response)
	}
	
	void destroy() {		
	}
	
	void init(FilterConfig filterConfig) {		
		this.filterConfig = filterConfig
	}
	
}

Project: PhotoHosting

Description

This is a very simple website offering uploading and sharing of 2 megapixel size photos, at a relatively high quality, social-media aware, and very hard to “steal”.

Motivation

A lot of free image hosting services are either overly ad-filled, or explicitly prevent uploading content which may be considered “NSFW”.

I wanted something less restrictive, less busy with ads (but still able to have ads on it), using some technologies which I would hope would facilitate fast I/O.

One principle was to try to always have the photo viewer coerced into visiting the website itself, as opposed to simply serving the image file.

This is accomplished by having the social media thumbnails relatively small, and a lower quality. Any attempt to actively open the image itself will always result in the webapp serving an entire page with the image embedded.

Technologies

I wrote this website is written in Java on the Apache Struts 2 framework accessing an HBase database.

Of the NoSQL databases, HBase seemed to be the only one which natively supports larger binary content, so it was a relatively easy option.

You can access the primary site here: https://photohosting.hostnucleus.ca/

To view an example of how the photos are presented: https://photohosting.hostnucleus.ca/show/cooked-chicken-on-white-plate-8de5b69c-020d-4480-addc-5d3af7013ba8.action

Update

Well, thanks to the complete cluster-fuck that is Java 11 (with it’s complete butchering of Java compatibility), and Apache’s lacklustre ability to move HBase away from Java 1.8, this project is officially dead. I simply cannot get the Tomcat webapp to talk to HBase.

So … yeah … fuck the OpenJDK “community”, and fuck Apache.

Project: Kalliope

Last year, I set myself the task of trying to actually produce a bunch of smaller web projects for s few reasons:

  • To learn some new frameworks
  • To illustrate my ability to learn there different frameworks
  • To showcase my skills in development in general

To that end, I thought I’d explain each project here in my blog, and the motivation behind each.

First up is Kalliope – https://kalliope.hostnucleus.ca

This is a “font server” with two primary end-goals – be able to arbitrarily upload a true type font, and have it produce web downloadable fonts, or to actively serve unlicensed fonts (in the same vein as Google Fonts).

The latter end-goal (as detailed above) was actually the primary goal on project inception – I wanted to have the ability to use a variety of fonts, but I didn’t want Google snooping the traffic on my sites (e.g. cookies, etc.). You can think of it as a replace for Google Fonts, really.

The other end-goal came about when the place where I worked had a client who had purchased some proprietary font, and wanted it used on their website. I didn’t want this font directly in the font server itself (as it requires a license), but all the leg-work had already been done to perform the conversion.

Interestingly enough, the original work on this was actually done for a completely different project – I had implemented a lot of the code within a project for generating print materials using SVGs. Since the SVGs needed to be displayable in all browsers, as well as the final PDF, I decided to use “FontForge” to do all the heavy lifting. It was only years later when developing a website where it occurred to me to break out the functionality into a separate web service.

It’s also a good project to learn and showcase the Grails framework.

Turning off Rails’ annoying “secret_key_base”

One of the most frustrating things with Rails is how they force shit down your throat when you don’t need or want it. Enter Rails “credentials” and their “securing” thereof. Where I work, we have no use for it. We don’t use Heroku or AWS. So why force me to set up a bunch of files or environment variables for something which I don’t want???

And setting “config.require_master_key = false” does not resolve the issue.

Well, here’s how I killed that filthy bitch:

In the file “/config/application.rb” add the following method:

There – now Rails can take their stupid “security” and shove it up their asses.

Alternative to Ruby’s const_defined? method

Today I wrote a bit of code in a base class which checks to see if a constant named “EXPORT_MAPPING” was defined. I’m not a Ruby expert, so off to Professor Google I go, and all the references I could find say to use the “const_defined?” method, thus resulting in the following implementation:

self.class.const_defined?(:EXPORT_MAPPING)

Now, since the intention here is to facilitate multiple levels of inheritance, I have a subclass called “ImageSlideshowComponent”, and a subclass of that “NivoSlideshowComponent”. ImageSlideshowComponent contains the EXPORT_MAPPING definition, with the intention of NivoSlideshowComponent simply inheriting it. Which is sort of does. We correctly see if it we do this:

NiveSlideshowComponent::EXPORT_MAPPING

However, invoking the following gives us “false”:

NiveSlideshowComponent.const_defined?(:EXPORT_MAPPING)

Clearly I’ve misunderstood the intention of the “const_defined?” method. It seems to only indicate if the class in question itself has defined the constant.

Here is the revised code block from the superclass which works correctly:

self.class.constants.include?('EXPORT_MAPPING')

Grails/Spring service autowiring

While working on a Grails project, I came across 2 interesting things:

  1. autowiring for domain classes doesn’t work, even if you follow all the instructions on how to enable it (either globally, or in the domain class definition itself)
  2. autowiring on abstract service classes doesn’t work (so if you’re implementing a data service with some custom functionality, your services won’t get autowired)

I’ve been able to fix both of these issues using a very simple utility class:

import grails.util.Holders

/**
 * Since Grails doesn't seem capable of autowiring into abstract classes, this
 * causes a bit of an issue when creating data services which do more than
 * just get/save.
 * 
 * All those context beans are available, so we just need to fudge the idea of
 * autowiring, and this class helps by providing a quick reference to the
 * singe long line required to get those Spring beans.
 * 
 * This also helps with domain classes, which have autowiring disabled, and
 * despite all the documentation saying the contrary, into which you cannot
 * autowire.
 * 
 * That said, this might prove to be a faster solution for use within domain
 * classes anyway since we don't have to worry about autowiring happening on
 * every object instantiation, but rather we just have a reference to the
 * singular instance with in the main context.
 */
class Beans {
	
	static def get(String name) {
		Holders.grailsApplication.mainContext.getBean(name)
	}
	
	static def $static_propertyMissing(String name) {
		get(name)
	}
	
}

Now, instead of setting up a service like this:

def myService

You instead define a getter method like this:

def getMyService() {
    Beans.myService
}

You can then use “myService” as you would’ve expected to under normal autowiring circumstances.

Gradle deployment script for Grails webapp

I haven’t had much success with finding useful deployment strategies and/or scripts for Grails anywhere. The extent of the documentation I’ve been able locate for deployment simply tells you to create a WAR and upload it to the servlet container.

Not terribly helpful if you want to run a formal process.

So, for my Grails webapps, I came up with this. I create a file in the “gradle” directory named “deploy.gradle” containing the following:

Also in the “gradle” directory is a subdirectory named “deploy” where I have the files specific to the environments to which I can deploy, such as “staging.gradle”:

Now, you will need to make some modifications to your main “build.gradle” script. First, add in the SSH plugin dependency, immediately after the “buildscript” section:

And then in the area where you have all the “apply” instructions:

Using the script above, I can deploy a particular branch from within my git repository to a specific environment thus:

It’s probably not perfect, but since I’m new to Gradle and Grails, I think it’s a pretty good start!

CM3 v5.10 on Ubuntu

This was a fairly difficult task to figure out (no documentation out there), so I thought I’d share how I ended up getting the latest version of the Critical Mass Modula-3 compiler installed.

Please note that I was working an an emulated 32-bit operating system, so if you’re working on a 64-bit installation, make changes where applicable.

  1. You must first download an actual binary copy of CM3. This isn’t actively mentioned anywhere in the source documentation, but you will notice that as you try to do any of the scripts, you will always fail before anything else happens. If you pay close attention, you will see the script is trying to run “cm3” … which is annoying, because it’s CM3 you’re trying to install :PGo to the URL https://modula3.elegosoft.com/cm3/ and scroll down to the section “Target Platform LINUXLIBC6”. I just downloaded the first file, so it’s the whole shebang.
  2. Extract the file and run “cminstall” – this will install the cm3 compiler in a place you wouldn’t necessary expect: /usr/local/cm3/bin – this means you won’t be able to invoke cm3 right away, so you must …
  3. Add the cm3 binary directory to the path thus:
  4. Next, clone the latest source respository:
  5. From the directory cm3/scripts, run the upgrade.sh script

And presto – your CM3 should now be upgraded all the way.

Update:

When trying to build all the additional packages (using “do-cm3-all.sh buildship“), even if you have ODBC drivers installed, the process will fail when building the “db” packages. To fix this:

This holds true for the UI package, as well, specifically lixXaw and libXmu:

 

Windows Server + IIS + PHP 5.3 + ImageMagick + PDFs

After 3 days and hours of frustration, here is the steps to follow for getting php_imagick working on Windows server running IIS:

  • make sure everything you install is 32-bit – this is because PHP is 32-bit, and introducing any 64-bit component will break the whole thing
  • download php_imagick here: http://windows.php.net/downloads/pecl/releases/imagick/3.1.2/php_imagick-3.1.2-5.3-nts-vc9-x86.zip
  • place only the php_imagick.dll file in the “ext” subdirectory of PHP
  • install a legacy version of ImageMagick – this is because as the forum thread http://stackoverflow.com/questions/8457744/installing-imagemagick-onto-xampp-windows-7 points out, PHP 5.3 is compiled with a different version of MSVC than the latest version of ImageMagick, and the two don’t talk to each other. Unfortunately, ImageMagick doesn’t keep archives of old versions, but thankfully other sites do: http://ftp.sunet.se/pub/multimedia/graphics/ImageMagick/binaries/ImageMagick-6.6.5-10-Q16-windows-dll.exe
    For simplicity sake, install the file in “C:\ImageMagick”
  • install a legacy version of GhostScript – again, this is to make sure the versions are compatible with each other. Again, Ghostscript themselves don’t seem to keep legacy files, but you can get them SourcrForge: http://sourceforge.net/projects/ghostscript/files/GPL%20Ghostscript/8.62/gs862w32.exe/download
    For simplicity sake, install the file in “C:\Ghostscript”
  • After all this, ImageMagick still won’t see Ghostscript as a delegate for handling PDF’s – you must edit the “config\delegates.xml” file and replace all instances of “@[email protected]” with the full path to the Ghostscript binary (note the forward slashes) as described http://stackoverflow.com/questions/13304832/ghostscripts-file-path-in-imagemagick: C:/Ghostscript/8.62/bin/gswin32c.exe
  • At this stage, any command-line testing of the ImageMagick to Ghostscript communication will probably work – but it won’t work under IIS. This is because by default, Ghostscript uses “C:\Windows\Temp”, but IIS doesn’t have permission to that directory. You must grant read/write access to that directory to “IIS_IUSRS” (or php_imagick will keep reporting that there is no delegate) as explained here: http://www.wizards-toolkit.org/discourse-server/viewtopic.php?f=1&t=24757&p=110439&sid=35e443f4faf1b92d68632a72c4000d3e#p110439
  • Finally – REBOOT! None of this will work without rebooting at least once so the entire OS has references to the newly installed software and libraries.

With any luck, you should now have a working php_imagick installed – you can confirm (at least part) of this using phpinfo() – although this may report php_imagick installed, it’s not a guarantee that PDF support will be working. You won’t know that part until you actually try to read in a PDF.