Htmlunit process is controlled from this servlet filter class

cc.kune.core.server.searcheable.SearchEngineServletFilter

Init method sets thread configuration:

  public void init(final FilterConfig filterConfig) throws ServletException {
    this.filterConfig = filterConfig;
    cache = new Cache();
    executor = Executors.newFixedThreadPool(THREADS);
  }

We have to define following issues to address a solution:

• WHEN have the server to avoid launching htmlunit? current server load %, current mem usage %...
• WHAT behavior will the server have in that case? response HTTP 404?, response HTTP 500? I think crawlers are very sensitive to this!

Enhancement #512: Page static generation via htmlunit perf issues

• Author: Pablo Ojanguren
• Status: New
• Priority: Normal
• Assignee: Pablo Ojanguren
• Category: Server side
• Target version: Unplanned
• Resolution:
• Tags:

Generation of static html pages for crawler using htmlunit is provoking a lack of performance sometimes. We want to avoid this controlling how and when htmlunit processes are executed.