7.1.8. groonga-suggest-httpd — Groonga v16.0.9 documentation

Skip to main content
`Ctrl`+`K`

# 7.1.8. `groonga-suggest-httpd`

## 7.1.8.1. Summary

`groonga-suggest-httpd` is a program that provides HTTP interface for the following features:

> - Returning Suggest execution result

> - Saving logs for learning

`groonga-suggest-httpd` provides suggest feature like suggest command. Note that some parameter names are different of them.

## 7.1.8.2. Syntax

`groonga-suggest-httpd` requires database path:

    groonga-suggest-httpd [options] DATABASE_PATH

## 7.1.8.3. Usage

You need to create one or more datasets to use `groonga-suggest-httpd`. A dataset consists of tables and columns. You can define them by groonga-suggest-create-dataset.

You need to use groonga-suggest-learner to learn suggestion data from user inputs. You doesn’t need to use groonga-suggest-learner when you create suggestion data by hand. See Suggest and sub documents about creating suggestion data by hand.

You can use `groonga-suggest-httpd` via HTTP after you create one or more datasets.

The following sections describes the followings:

> - How to set up a dataset

> - How to use `groonga-suggest-httpd` with groonga-suggest-learner

> - How to use `groonga-suggest-httpd` for retrieving suggestions.

### 7.1.8.3.1. Setup

You need to create a dataset by groonga-suggest-create-dataset.

Here is an example that creates `query` dataset:

Execution example:

    $ groonga-suggest-create-dataset ${DB_PATH} query
    > plugin_register suggest/suggest
    true
    > table_create event_type TABLE_HASH_KEY ShortText
    true
    > table_create bigram TABLE_PAT_KEY ShortText --default_tokenizer TokenBigram --normalizer NormalizerAuto
    true
    > table_create kana TABLE_PAT_KEY ShortText --normalizer NormalizerAuto
    true
    > table_create item_query TABLE_PAT_KEY ShortText --default_tokenizer TokenDelimit --normalizer NormalizerAuto
    true
    > column_create bigram item_query_key COLUMN_INDEX|WITH_POSITION item_query _key
    true
    > column_create item_query kana COLUMN_VECTOR kana
    true
    > column_create kana item_query_kana COLUMN_INDEX item_query kana
    true
    > column_create item_query freq COLUMN_SCALAR Int32
    true
    > column_create item_query last COLUMN_SCALAR Time
    true
    > column_create item_query boost COLUMN_SCALAR Int32
    true
    > column_create item_query freq2 COLUMN_SCALAR Int32
    true
    > column_create item_query buzz COLUMN_SCALAR Int32
    true
    > table_create pair_query TABLE_HASH_KEY UInt64
    true
    > column_create pair_query pre COLUMN_SCALAR item_query
    true
    > column_create pair_query post COLUMN_SCALAR item_query
    true
    > column_create pair_query freq0 COLUMN_SCALAR Int32
    true
    > column_create pair_query freq1 COLUMN_SCALAR Int32
    true
    > column_create pair_query freq2 COLUMN_SCALAR Int32
    true
    > column_create item_query co COLUMN_INDEX pair_query pre
    true
    > table_create sequence_query TABLE_HASH_KEY ShortText
    true
    > table_create event_query TABLE_NO_KEY
    true
    > column_create sequence_query events COLUMN_VECTOR|RING_BUFFER event_query
    true
    > column_create event_query type COLUMN_SCALAR event_type
    true
    > column_create event_query time COLUMN_SCALAR Time
    true
    > column_create event_query item COLUMN_SCALAR item_query
    true
    > column_create event_query sequence COLUMN_SCALAR sequence_query
    true
    > table_create configuration TABLE_HASH_KEY ShortText
    true
    > column_create configuration weight COLUMN_SCALAR UInt32
    true
    > load --table configuration
    > [
    > {"_key": "query", "weight": 1}
    > ]
    1

groonga-suggest-create-dataset outputs executed commands. You can confirm that what tables and columns are created for the new dataset.

### 7.1.8.3.2. Launch `groonga-suggest-learner`

You can choose whether you use learned suggestion data immediately or not.

There are two ways to use learned suggestion data immediately:

> - Both of `groonga-suggest-httpd` and groonga-suggest-learner use the same database

> - `groonga-suggest-httpd` receives learned suggestion data from groonga-suggest-learner

In the former case, you must run both `groonga-suggest-httpd` and groonga-suggest-learner on the same host.

In the latter case, you can run `groonga-suggest-httpd` and groonga-suggest-learner on different hosts.

If you don’t need to use learned suggestion data immediately, you need to apply learned suggestion data from database that is used by groonga-suggest-learner to database that is used by `groonga-suggest-httpd` by hand. Normally, this usage is recommended. Because learned suggestion data may have garbage data by inputs from evil users.

In this document, learned suggestion data are used immediately by receiving learned suggestion data from groonga-suggest-learner. Both `groonga-suggest-httpd` and groonga-suggest-learner are running on the same host. Because it’s easy to explain.

Here is an example that launches groonga-suggest-learner. You need to specify database that has `query` dataset. This document omits the instruction for creating `query` dataset:

Execution example:

    $ groonga-suggest-learner --daemon ${DB_PATH}

The `groonga-suggest-learner` process opens two endpoints at `1234` port and `1235` port:

> - `1234` port: Endpoint that accepts user input data from `groonga-suggest-httpd`

> - `1235` port: Endpoint that sends learned suggestion data to `groonga-suggest-httpd`

### 7.1.8.3.3. Launch `groonga-suggest-httpd`

You need to launch `groonga-suggest-httpd` for the following proposes:

> - Learning suggestion data from user inputs

> - Providing suggestion result to clients

Here is an example that launches `groonga-suggest-httpd` that communicates groonga-suggest-learner:

Execution example:

    $ groonga-suggest-httpd --send-endpoint 'tcp://127.0.0.1:1234' --receive-endpoint 'tcp://127.0.0.1:1235' --daemon ${DB_PATH}

The `groonga-suggest-httpd` process accepts HTTP requests on `8080` port.

If you want to save requests into log file, use `--log-base-path` option.

Here is an example to save log files under `logs` directory with `log` prefix for each file:

    % groonga-suggest-httpd --log-base-path logs/log ${DB_PATH}

`groonga-suggest-httpd` creates log files such as `logYYYYmmddHHMMSS-00` under `logs` directory.

### 7.1.8.3.4. Learn from user inputs

You can learn suggestion data from user inputs.

You need to specify the following parameters to learn suggestion data:

> - `i`: The ID of the user (You may use IP address of client)

> - `l`: The dataset name

> - `s`: The timestamp of the input in seconds

> - `t`: The query type (It’s optional. You must specify `submit` only when the user input is submitted.)

> - `q`: The user input

Here are example requests to learn user input “Groonga” in `query` dataset:

    .. groonga-command
    .. include:: ../../example/reference/executables/groonga-suggest-httpd/learn.log
    .. % curl 'http://localhost:8080/?i=127.0.0.1&l=query&s=92619&q=G'
    .. % curl 'http://localhost:8080/?i=127.0.0.1&l=query&s=93850&q=Gr'
    .. % curl 'http://localhost:8080/?i=127.0.0.1&l=query&s=94293&q=Gro'
    .. % curl 'http://localhost:8080/?i=127.0.0.1&l=query&s=94734&q=Groo'
    .. % curl 'http://localhost:8080/?i=127.0.0.1&l=query&s=95147&q=Grooon'
    .. % curl 'http://localhost:8080/?i=127.0.0.1&l=query&s=95553&q=Groonga'
    .. % curl 'http://localhost:8080/?i=127.0.0.1&l=query&s=95959&t=submit&q=Groonga'

Inputting data must not use `t=submit` parameter. In the above example, you just learn user inputs but you can learn and get complete candidates at once. It’s described at the next section.

Submitted data must use `t=submit` parameter.

### 7.1.8.3.5. Use suggested response

You can get suggested result from `groonga-suggest-httpd`.

You need to specify the following parameters to get suggested result:

> - `n`: The dataset name

> - `t`: The query type (`complete`, `correct` and/or `suggest`)

> - `q`: The user input

You can also specify parameters for suggest as option.

Here is an example that gets Completion result. The result is computed by using learned data at the previous section. `frequency_threshold=1` parameter is used because this is an example. The parameter enables input data that are occurred one or more times. Normally, you should not use the parameter for production. The parameter will increase noises:

Execution example:

    $ curl 'http://localhost:8080/?n=query&t=complete&q=G&frequency_threshold=1'
    {"complete":[[1],[["_key","ShortText"],["_score","Int32"]],["groonga",1]]}

You can combine completion and learning by specifying parameters for both:

Execution example:

    $ curl 'http://localhost:8080/?i=127.0.0.1&l=query&s=96000&q=G&n=query&t=complete&frequency_threshold=1'
    {"complete":[[1],[["_key","ShortText"],["_score","Int32"]],["groonga",1]]}

## 7.1.8.4. Command line parameters

### 7.1.8.4.1. Required parameters

There is only one required parameter.

#### 7.1.8.4.1.1. `DATABASE_PATH`

Specifies the path to a Groonga database. This database must have one or more datasets. Each dataset must be created by groonga-suggest-create-dataset.

### 7.1.8.4.2. Optional parameters

-p, --port

  Specify HTTP server port number.

  The default port number is `8080`.

-t, --n-threads

  Specify number of threads.

  This option accepts `128` as the max value, but use the number of CPU cores for performance.

  The default value is the number of CPU cores.

-s, --send-endpoint

  Specify endpoint URI of groonga-suggest-learner for sending user inputs.

  The format is `tcp://${HOST}:${PORT}` such as `tcp://192.168.0.1:2929`.

  The default value is none.

-r, --receive-endpoint

  Specify endpoint URI of groonga-suggest-learner for receiving learned suggestion data.

  The format is `tcp://${HOST}:${PORT}` such as `tcp://192.168.0.1:2929`.

  The default value is none.

-l, --log-base-path

  Specify path prefix of log.

  The default value is none.

--n-lines-per-log-file

  Specify the max number of lines in a log file.

  The default value is `1000000`.

-d, --daemon

  Specify this option to daemonize.

  Don’t daemonize by default.

--disable-max-fd-check

  Specify this option to disable checking the max number of file descriptors on start.

  Check by default.

## 7.1.8.5. GET parameters

`groonga-suggest-httpd` accepts some GET parameters.

There are required parameters which depend on query type.

In `complete`, `correct` or `suggest` query type, unhandled parameters are passed through suggest. It means that you can use parameters of suggest.

### 7.1.8.5.1. Required parameters

You must specify the following parameters.

Key | Description | Note
--- | --- | ---
`q` | Input by user. It must be UTF-8 encoded string. |

### 7.1.8.5.2. Required parameters for learning

You must specify the following parameters when you specify `--send-endpoint`.

Key | Description | Note
--- | --- | ---
`s` | Elapsed time since `1970-01-01T00:00:00Z`. | The unit is millisecond.
`i` | Unique ID to distinct each user | Session ID, IP address and so on will be usable for this value.
`l` | One or more learn target dataset names. You need to use `|` as separator such as `dataset1|dataset2|dataset3`. | Dataset name is the name that you specify to groonga-suggest-create-dataset.

### 7.1.8.5.3. Required parameters for suggestion

You must specify the following parameters when you specify one of `complete`, `correct` and `suggest` to `t` parameter.

Key | Description | Note
--- | --- | ---
`n` | The dataset name to use computing suggestion result. | Dataset name is the name that you specify to groonga-suggest-create-dataset.
`t` | The query type. Available values are `complete`, `correct`, `suggest`. | You can specify multiple types. You need to use `|` as separator such as `complete|correct`.

### 7.1.8.5.4. Optional parameters

Here are optional parameters.

Key | Description | Note
--- | --- | ---
`callback` | Function name for JSONP |

### 7.1.8.5.5. Optional parameters for learning

Here are optional parameters when you specify `--send-endpoint`.

Key | Description | Note
--- | --- | ---
`t` | The query type. Available value is only `submit`. | You must specify `submit` when user submits the input specified as `q`. You must not specify `submit` for user inputs that aren’t submitted yet. You can use suggestion by specifying `complete`, `correct` and/or `suggest` to `t` when you doesn’t specify `submit`. See Required parameters for suggestion for details about these values.

## 7.1.8.6. Return value

`groonga-suggest-httpd` returns the following format response. It’s the same format as body of suggest:

    {
      TYPE1: [
        [CANDIDATE_1, SCORE_1],
        [CANDIDATE_2, SCORE_2],
        ...,
        [CANDIDATE_N, SCORE_N]
      ],
      TYPE2: [
        [CANDIDATE_1, SCORE_1],
        [CANDIDATE_2, SCORE_2],
        ...,
        [CANDIDATE_N, SCORE_N]
      ],
      ...
    }

Here is the response when `t` is `submit`:

    {}

### 7.1.8.6.1. `TYPE`

One of `complete`, `correct` and `suggest`.

### 7.1.8.6.2. `CANDIDATE_N`

The string of candidate in UTF-8.

### 7.1.8.6.3. `SCORE_N`

The score of the candidate.

Candidates are sorted by score descendant.

## 7.1.8.7. See also

> - suggest