Tuesday, February 15, 2011

Ajax-solr, struts2 and a backend solr server

I'd been using solr for a while when I stumbled across the ajax-solr project. Its really quite neat, well thought out and is relatively easy to customise. Within a few hours I had it integrated into our webapp and running queries with beautifully paginated & highlighted results.
All well and good, until I started to think about to roll it out to the production environment. The ajax-solr library is fast and works well as it communicates directly with the solr server... but in our case this is several tiers down and not accessible to the outside world. Solr itself has no security built in and the thought of trying to convince the network security guys to open up a route through to the solr server was not even worth entertaining.
There are a few postings on how to write a proxy servlet to pass requests through but they all looked too complicated and exposed too many risks. I was looking at having to throw away the investment in ajax-solr and roll my own server based solution.
However, for this webapp we use Struts2 to handle some of the requests and handle ajax requests through the actions. Why not set the ajax-solr url to point to a Struts2 url, within the action make the request to the backend solr server and then stream the results back ? Turned out to be simpler to do than write the previous sentence.
First I had to change the way the AjaxSolr.Manager crafts the url as it wasn't going to work as it was -
Changed a line in Manager.jquery.js from
jQuery.getJSON(this.solrUrl + servlet + '?' + this.store.string() + '&wt=json&json.wrf=?', {}, function (data) { self.handleResponse(data); });
to
jQuery.getJSON(this.solrUrl + '?' + this.store.string() + '&wt=json&jsonwrf=?', {}, function (data) { self.handleResponse(data); });
to rename the parameter "json.wrf" to "jsonwrf" (its not easy to handle a parameter name with a dot in Struts2... or at least I didn't even try) and remove the "servlet" part as I needed to construct a url in the form "myaction!solr.action?q...."

Then add to my action a solr method a bit like this -

 
    private String q, solrrequest,jsonwrf;
    private Integer start;

    public String solr() throws Exception {
        logger.debug("Original request - " + getRequest().getQueryString());
        
        StringBuffer query = new StringBuffer();
        query.append("q=");
        query.append(URLEncoder.encode(q, "UTF-8"));
        if (start != null) {
            query.append("&start=");
            query.append(start);         
        }
        query.append("&hl=on");
        query.append("&hl.fl=*");
        query.append("&wt=json");
        if (jsonwrf != null) {
            query.append("&json.wrf=");
            query.append(jsonwrf);         
        }
        
        solrrequest = "http://mysolrserver/solr/select?" + query.toString();

        logger.info("Built Http Get request to solr: " + solrrequest);

        return "jsondata";
    }

    public InputStream getInputStream() throws Exception {
        HttpClient httpClient = new HttpClient();

        GetMethod method = new GetMethod(solrrequest);

        int statusCode = httpClient.executeMethod(method);

        logger.info("Http Get request returned response: '" + HttpStatus.getStatusText(statusCode) + "'");

        return method.getResponseBodyAsStream();

    }


then in my struts.xml add a mapping a bit like this which makes the output a stream of json data from the getInputStream() method above.

 
        <action name="myaction" class="com.me.MyAction">  
            <result name="jsondata" type="stream">
                <param name="contentType">application/json</param>
                <param name="inputName">inputStream</param>
            </result>            
        </action>                 


and finally the setup of the ajax-solr looks a bit like this -

 
  $(function () {
   
    Manager = new AjaxSolr.Manager({
     solrUrl: 'myaction!solr.action'
    });
    
    Manager.addWidget(new AjaxSolr.ResultWidget({
        id: 'result',
        target: '#results'
      }));

    .
    . 
    .



So we have ajax-solr talking through a Struts2 action to the backend solr server. The advantage being that the access to the solr server is constrained by what you put in the action and there is only a moderate degradation in performance.

Hope that helps someone one day solve a similar problem.

No comments: