Date range search

Purpose

Date range search supports to aggregate (count the number of documents in a search result) per date range, and supports to filter the search result on date ranges. The support does not include ranking on date, for example to prefer more recent documents over older documents. This is currently unsupported and may be added to Enterprise Search in the future.

Field doc_created

Field doc_created is used by default to compare against date ranges.

• Documents always include this Dublin Core field.
• All Enterprise Search providers attempt to populate this field.
• The field represents the document creation date, not a last update date.

It is possible to use another field by setting property Field of a search query aggregation class; a class derived from class SearchQueryBaseAggregation.

Aggregation

Aggregation requests can be added to a search query, class SearchQuery. This requests to count the number of documents, per aggregation entry, within the search result.

A date range aggregation can be used, class SearchQueryDateRangeAggregation. The aggregation accepts a number of date ranges, class SearchQueryDateRange. A date range includes:

• A key, for example "Previous 5 calendar years". The key identifies the date range. The key is sent to Elastic Search. The key could be used for presentation purposes.
• A from-specification, for example with an operand "now-5y/y". The operand is an Elastic Search date math expression, see below. The from-specification can be omitted.
• A to-specification, for example with an operand "now/y". The to-specification can be omitted.

Each date range results in a count. Date ranges can overlap. With overlapping date ranges documents can be counted more than once.

Date ranges and their counts can be presented in addition to the search result. Checkboxes can be used to select date ranges. Selected date ranges can be used to filter the search result in a next search.

Filtering

Selected date ranges can be added to the search query, as date range filters, class DateRangeFilterValue. The filter value includes

• A key, for example "Previous 5 calendar years". The key is for use by Enterprise Search. Enterprise Search does not send the key to Elastic Search.
• Key-value pairs used to specify the date range filter.

Key-value pair examples are:

• "gte", "now-5y/y": operator greater than or equal to, and again a date math expression
• "lt", "now/y", operator less than, and a date math expression.

Date format

An Elastic Search date math expression may include an absolute date, for example "1/12/2022". A date format specification is required to tell whether this is December 1 or January 12. Enterprise Search fixes the date format and demands "2022-12-01" in this case. It does so by adding a key-value pair to the above date range filter:

• "format", "yyyy-MM-dd": 4-digit year, 2 digit month, 2 digit day.

Missing values

Documents that lack a value for doc_created are normally not counted in the aggregation. Enterprise Search specifies a date to be used if a date value is missing. ES establishes this date as follows:

• the from-date of the first date range in the aggregation specification
• else the to-date of the first date range
• else the from-date or next the to-date in de following date ranges, until a value is established.

A suitable date is the earliest date in all aggregation date ranges. It is therefore advisable the specify the date ranges in order, starting with the earliest date range.

ES applies this approach for missing values to doc_created or to the date field used instead.

Date math

A date math expression is for example "now/y": the current date, truncated down to the calendar year. For example if the current date is 2022-12-15 the result is 2022-01-01.

A meaningful combination of date ranges for aggregations could be:

• Range for documents older than 5 years: no from-specification, only a to-specification: "now-5y/y". All documents before 2017-01-01.
• Range for the past 5 calendar years: from-specification: "now-5y/y", to-specification: "now/y". All documents from 2017-01-01, included, to 2022-01-01, excluded.
• Range for this year: from-specification: "now/y", no to-specification. All documents in this running year, starting with 2022-01-01, included.

Enterprise Search passes date math expressions to Elastic Search. Refer to their date math documentation.

Filter specified by key only

Date math expressions are used in aggregations and in filters.

• An aggregation date range uses a from-specification with a date math expression, and/or a to-specification with a date math expression.
• A filter date range uses operators gt, gte, lt or lte, with a date math expression.

It is possibly to aggregate only, or to filter only, or the usual case to aggregate and to filter. Aggregating and filtering are independent, requiring to specify date math expressions for aggregations and requiring to specify - often the same - date math expressions in the filter.

Enterprise Search supports to specify the filter by key only. It implements the following logic:

• if the filter is empty, that is: contains no gt, gte, lt or lte operators
• and the filter key matches the key of an aggregation date range
• then use the date math expression in the from-specification of the aggregation as a gte (greater than or equal to) compare in the filter
• and use the date math expression in the to-specification of the aggregation as a lt (less than) compare in the filter.