make index file sparse (no null props)

[nkuehn]

Hi,
the index.json file size is a critical limitation of this great plugin.
In my text index (which is too big with nearly 2MB) I saw that every document has a data structure.

    "1": {
      "id": 1,
      "title": "Category Examples Java",
      "url": "/complete-code-samples/java/CategoryExamples.html",
      "date": null,
      "categories": null,
      "tags": null,
      "is_post": false
    },

Including null values is inefficient, in size critical JSON it's better to completely omit the property.

Could be breaking to the client javascript, the client would have to be robust against missing keys (but maybe it is).

[slashdotdash]

Will gladly accept a pull request if you are able to make this change?

One possible solution would be to only include properties in the document that are non-nil in indexer.rb.

[nkuehn]

sure :-)

I consider experimenting in a branch that uses lunr 2.0.x since that has a more space efficient index structure by design that is a more fundamental improvement than the incremental one this issue is about. https://github.com/olivernn/lunr.js/blob/master/CHANGELOG.mdown#200

Is there already an initiative for that? I couldn't find an issue, branch or fork.

Pretty limited free time though at the moment.

[slashdotdash]

@nkuehn I have nothing planned for lunr 2.0 so any contribution you can make is appreciated.

[nkuehn]

see PR :-)