Spider REST Commands : Object Update Commands : Add Batch
Add Batch
A batch of new objects is added to a specific table in an application using the following REST command:
POST /{application}/{table}
where {application} is the application name and {table} is the table in which the objects are to be added. If the given table name does not exist but the application’s AutoTables option is true, the table is implicitly created before the batch is added. If AutoTables is false and the table is unknown, an error is returned.
The command must include an input entity that contains the objects to be added. The format of an example input message in XML as shown below:
<batch>
<docs>
<doc>
<field name="_ID">AAFE9rf++BCa3bQ4HgAA</field>
<field name="SendDate">2010-07-18 05:22:35</field>
<field name="Size">3654</field>
<field name="Subject">RE: synopses copens silk seagull citizens</field>
<field name="Tags">
<add>
<value>AfterHours</value>
<value>Customer</value>
</add>
</field>
<field name="InternalRecipients">
<add>
<value>KMTkYYrkL4MmrHxj//ZVhQ==</value>
</add>
</field>
<field name="Sender">wWR7yZik2p1rrI6/qSepXg==</field>
...
</doc>
<doc>
<field name="_ID">AAFE9rf++BCa3bQ4HgAB</field>
<field name="SendDate">2010-07-17 10:37:03</field>
<field name="Size">15830</field>
<field name="Subject">bloodily douches spadones poinds vestals</field>
<field name="Tags">AfterHours</field>
<field name="InternalRecipients">Ii91O7qHb8rPhzvaihzTqw==</field>
<field name="Sender">fiJOQPhAJqQJeuEOD+iH8Q==</field>
...
</doc>
...
</docs>
</batch>
In JSON:
{"batch": {
"docs": [
{"doc": {
"_ID": "AAFE9rf++BCa3bQ4HgAA",
"SendDate": "2010-07-18 05:22:35",
"Size": "3654",
"Subject": "RE: synopses copens silk seagull citizens",
"Tags": {
"add": ["AfterHours", "Customer"]
},
"InternalRecipients": {
"add": ["KMTkYYrkL4MmrHxj//ZVhQ=="]
},
"Sender": "wWR7yZik2p1rrI6/qSepXg==",
...
}},
{"doc": {
"_ID": "AAFE9rf++BCa3bQ4HgAB",
"SendDate": "2010-07-17 10:37:03",
"Size": "15830",
"Subject": "bloodily douches spadones poinds vestals",
"Tags": "AfterHours",
"InternalRecipients": "Ii91O7qHb8rPhzvaihzTqw==",
"Sender": "fiJOQPhAJqQJeuEOD+iH8Q==",
...
}},
...
]
}}
Semantics about adding batches are described below:
•
Object ID: If the _ID field is not defined for an object, Spider assigns a unique ID based (e.g., “AAFE95dAqRCa3bQ4HgAA”). The ID is a base 64-encoded 120-bit value that is generated in a way to ensure uniqueness even in a multi-node cluster.
•
MV scalars and link fields: When an MV scalar or link field is assigned a single value, it can use the same syntax as an SV scalar field. For example, the link Sender is assigned a single value and uses the simplified name/value syntax. But when an MV scalar or link field is assigned multiple values, they must be enclosed in an add group. See for example the MV scalar Tags, which is assigned 2 values. An MV field can always use the add group syntax even if it being assigned a single value. See for example the link field InternalRecipients.
•
Batch size: Batches can be arbitrarily large, but large batches require more memory since the entire parsed batch is held in memory. Also, mutations are flushed when their count reaches batch_mutation_threshold, defined in doradus.yaml. Hence, large batches can be stored as multiple “sub-batch” transactions.
•
Existing IDs: If an object in an Add Batch command is given an ID that corresponds to an existing object, the existing object is updated. This preserves the idempotent update semantics of Add Batch commands: if the same object is added twice, the second “add” is treated as an “update” and, since all of the values will be the same, the second “add” will be a no-op.
•
Binary fields: Binary field values must be encoded as declared using the encoding declared in the schema (Base64 or Hex).
•
Group fields: Group fields can be ignored, and assignments can be made to leaf fields directly. However, leaf fields can also be qualified via the owning group. For example, in the doc group below, Participants and Recipients are both group fields:
<doc>
<field name="Participants">
<field name="Recipients">
<field name="InternalRecipients">KMTkYYrkL4MmrHxj//ZVhQ==</field>
</field>
<field name="Sender">wWR7yZik2p1rrI6/qSepXg==</field>
</field>
...
</doc>
If the Add Batch command is successful, the request returns a 201 Created response. The response contains a doc element for each object in the batch:
<batch-result>
<status>OK</status>
<has_updates>true</has_updates>
<docs>
<doc>
<updated>true</updated>
<status>OK</status>
<field name="_ID">AAFE9rf++BCa3bQ4HgAA</field>
</doc>
<doc>
...
</doc>
</docs>
</batch-result>
In JSON:
{"batch-result": {
"status": "OK"
"has_updates": "true",
"docs": [
{"doc": {
"updated": "true",
"status": "OK",
"_ID": "AAFE9rf++BCa3bQ4HgAA"
}},
{"doc": {
...
}}
]
}}
As shown, a doc element is given with an _ID value for every object added or updated including objects whose _ID was set by Doradus. The status element indicates if the update for that object was a valid request, and the updated element indicates if a change was actually made to the database for that object. If at least one object in the batch was updated, the has_updates element is included with a value of true. If an individual object update failed, its status will be Error, and an error message will be included in its doc element. If no objects in the batch were updated (because the existing objects already existed and had the requested values), the has_updates element will be absent. For example:
<batch-result>
<status>OK</status>
<docs>
<doc>
<updated>false</updated>
<status>OK</status>
<comment>No updates made</comment>
<field name="_ID">AAFE9rf++BCa3bQ4HgAA</field>
</doc>
<doc>
...
</doc>
</docs>
</batch-result>
In JSON:
{"batch-result": {
"status": "OK"
"docs": [
{"doc": {
"updated": "false",
"status": "OK",
"comment": "No updates made",
"_ID": "AAFE9rf++BCa3bQ4HgAA"
}},
{"doc": {
...
}}
]
}}
If the entire batch is rejected, e.g., due to a syntax error in the parsed input message, a 400 Bad Request is returned along with a plain text message. For example:
HTTP/1.1 400 Bad Request
Content-length: 62
Content-type: Text/plain
 
Child of link field update node must be 'add' or 'remove': foo