Xalok Go Documentation

Appearance

Import API integration ​

Introduction ​

In Xalok many entities have metadata that allow to adapt to multiple situations, so there is no previous complete specification (about such metastructures) that we can provide.

New templates, custom modules, special configurations, ... can be added and can be exported and imported. We will see how we can identify the particular structures for each project and adapt to them.

Definitions ​

  • Authentication server: the authentication server is normally available at https://www.myproject.com/oauth/v2
  • Import API: the import API server is normally available at https://admin.myproject.com/api/importer
  • User: the user alias to connect to the API. This is a normal Xalok user.
  • Password: the user's password in Xalok.
  • Client ID: even if the user exists, a request must be made (to those managing the application) to create a client for the API. This is the client id.
  • Secret ID: together with the client id will indicate a secret id.

Testing setup ​

For convenience and reproducibility, we define the following environment variables:

shell
IAPI_AUTH_SERVER=https://www.myproject.com/oauth/v2
IAPI_SERVER=https://admin.myproject.com/api/importer
IAPI_USER=my.user
IAPI_PASS=my.user.password
IAPI_CLIENT_ID=1234567890ABCDEFGHIJK
IAPI_SECRET_ID=abcdefghijk1234567890

Authentication ​

OAuth authentication ​

The authentication operation consists of obtaining a token. This operation can be specified with the following command:

shell
curl \
  -s \
  -G \
  -d "grant_type=password" \
  -d "client_id=$IAPI_CLIENT_ID" \
  -d "client_secret=$IAPI_SECRET_ID" \
  -d "username=$IAPI_USER" \
  -d "password=$IAPI_PASS" \
  "$IAPI_AUTH_SERVER/token"

With output:

json
{
  "access_token": "ODVlN...lMmMwNg",
  "expires_in": 3600,
  "token_type": "bearer",
  "scope": null,
  "refresh_token": "Mzg5...TY2Yw"
}

OAuth refresh token ​

Before expires_in you should refresh your token. Assuming $IAPI_REFRESH_TOKEN contains the refresh_token value. This operation can be specified with the following command:

shell
curl \
  -s \
  -G \
  -d "grant_type=refresh_token" \
  -d "client_id=$IAPI_CLIENT_ID" \
  -d "client_secret=$IAPI_SECRET_ID" \
  -d "refresh_token=$IAPI_REFRESH_TOKEN" \
  "$IAPI_AUTH_SERVER/token"

with output

json
{
  "access_token": "MmIw...diNw",
  "expires_in": 3600,
  "token_type": "bearer",
  "scope": null,
  "refresh_token": "ZDdmO...RjZA"
}

Full authentication / refresh example ​

shell
echo "Testing autentication..."
IAPI_AUTH_RES=`curl \
  -s \
  -G \
  -d "grant_type=password" \
  -d "client_id=$IAPI_CLIENT_ID" \
  -d "client_secret=$IAPI_SECRET_ID" \
  -d "username=$IAPI_USER" \
  -d "password=$IAPI_PASS" \
  "$IAPI_AUTH_SERVER/token"`
echo "JSON response:"
echo "$IAPI_AUTH_RES" | jq .
echo "Reading the token..."
IAPI_TOKEN=`echo "$IAPI_AUTH_RES" | jq -r .access_token`
echo "(TOKEN=$IAPI_TOKEN)"
echo "Reading the refresh token..."
IAPI_REFRESH_TOKEN=`echo "$IAPI_AUTH_RES" | jq -r .refresh_token`
echo "(REFRESH_TOKEN=$IAPI_REFRESH_TOKEN)"
echo "Testing refreshing token..."
IAPI_REFRESH_RES=`curl \
  -s \
  -G \
  -d "grant_type=refresh_token" \
  -d "client_id=$IAPI_CLIENT_ID" \
  -d "client_secret=$IAPI_SECRET_ID" \
  -d "refresh_token=$IAPI_REFRESH_TOKEN" \
  "$IAPI_AUTH_SERVER/token"`
echo "JSON response:"
echo "$IAPI_REFRESH_RES" | jq .
echo "Reading the new token..."
IAPI_TOKEN=`echo "$IAPI_REFRESH_RES" | jq -r .access_token`
echo "(new TOKEN=$IAPI_TOKEN)"

with output

text
Testing autentication...
JSON response:
{
  "access_token": "MmU4Y...RiNQ",
  "expires_in": 3600,
  "token_type": "bearer",
  "scope": null,
  "refresh_token": "ZmU0M...DcwNw"
}
Reading the token...
(TOKEN=MmU4Ym...WRiNQ)
Reading the refresh token...
(REFRESH_TOKEN=ZmU0...DcwNw)
Testing refreshing token...
JSON response:
{
  "access_token": "MjBi...IzMA",
  "expires_in": 3600,
  "token_type": "bearer",
  "scope": null,
  "refresh_token": "MjZ...kwMA"
}
Reading the new token...
(new TOKEN=MjBi...DIzMA)

Checking connection ​

We can check the import API server status and version with:

shell
curl \
  -s \
  -H "Authorization: Bearer $IAPI_TOKEN" \
  "$IAPI_SERVER/version"

with output

xml
<?xml version="1.0"?>
<item>
  <success>1</success>
  <version>1.0</version>
</item>

Export ​

The export process is important because with it we can easily obtain the appropriate import structures for any type of template, module, configuration, ...

Some operations do not require parameters to be known beforehand, such as exporting categories:

shell
curl \
  -s \
  -H "Authorization: Bearer $IAPI_TOKEN" \
  "$IAPI_SERVER/categories"

with output

xml
<?xml version="1.0"?>
<item>
    <item key="6">
        <data>
            <title>Default cat</title>
            <active>1</active>
            <slug>default-cat</slug>
            <description>Default cat</description>
            <id>6</id>
        </data>
        <type>category</type>
    </item>
    <item key="7">
        <data>
            <title>Cocina</title>
            <active>1</active>
            <slug>cocina</slug>
            <description>Cocina: arroces, pescados, postres, carnes, ...</description>
            <type>blog</type>
    ...

Other operations require knowledge of the entity's internal Xalok code:

shell
curl \
  -s \
  -H "Authorization: Bearer $IAPI_TOKEN" \
  "$IAPI_SERVER/images/3"

with output

xml
<?xml version="1.0"?>
<item>
    <data>
        <name>6241c9c45fde7.jpeg</name>
        <contentType>image</contentType>
        <title>Bici</title>
        <description>6241c9c45fde7.jpeg</description>
        <alt>Bici</alt>
        <base64file>/9j/4AAQSkZJRgABAQEAY...ov+FW6h/wA=</base64file>
    </data>
    <metadata/>
    <type>image</type>
</item>

Import ​

Except for certain properties such as the internal Xalok's id or the uuid which may not be repeated, any entity exported in a certain instance may be imported as is in another instance (or even in the same one).

The following example exports the previous image 3 and, as it has no uuid, we only need to add one (if we want) to import it again as another image within the same instance:

shell
curl \
  -s \
  -H "Authorization: Bearer $IAPI_TOKEN" \
  "$IAPI_SERVER/images/3" \
| sed 's/<data>/<data><uuid>'`date +%s`'<\/uuid>/g' \
| curl \
    -s \
    -X POST \
    -H "Authorization: Bearer $IAPI_TOKEN" \
    -H "Content-Type: application/xml" \
    -d @- \
    "$IAPI_SERVER/images"

with output

xml
<?xml version="1.0"?>
<item>
    <success>1</success>
    <imageId>1195</imageId>
</item>

getting the id that Xalok has assigned to it.

Automatic ID mapping ​

Within the metadata, Xalok can refer to other entities, for example within a news item, an image can be identified as:

shell
curl \
  -s \
  -H "Authorization: Bearer $IAPI_TOKEN" \
  "$IAPI_SERVER/articles/3177"
xml
<?xml version="1.0"?>
<item>
    <data>
        <template>default</template>
        <category>bazar</category>
        <title>Concurso de tartas de chocolate</title>
        <status>published</status>
        <publishedAt>2022-04-29T09:36:03+02:00</publishedAt>
    </data>
    <modules>
        ...
        <__contentModels>
            <type>image</type>
            <id>3</id>
        </__contentModels>
        ...

In order not to have to know or save in advance the codes of the entities we have already imported, it is possible to reference them using the uuid, for example for images:

xml
<__contentModels>
    <type>image</type>
    <id>uuid:image:83c1142-9450-4baf-8731-ef42cde1f13f</id>
</__contentModels>

ID mapping names ​

Currently the automatic ID mapping works for:

  • Pages: uuid:page:{sourceId} (a "page" is a general content grouping all specific ones: news, galleries, recipes, ...)
  • Images: uuid:image:{sourceId}
  • Videos: uuid:video:{sourceId}
  • Users: uuid:user:{slug}
  • Categories: uuid:category:{slug}

Note that the value of {uuid} (i.e. sourceId) need not follow the convention of OSF or any other organization, but should be unique strings for their scope across your ecosystem.

Autoroles ​

An important and ubiquitous element in Xalok are the roles that determine which elements are included in certain content as well as their order. You can specify the list of roles and then the modules in any order, or you can omit the list of roles, set the autoroles element, and enter the modules in the correct order.

xml
  <modules>
    <__roles>sponsor</__roles>
    <__roles>title</__roles>
    <__roles>epigraph</__roles>
    <__roles>main_image--1</__roles>
    <__roles>paragraph</__roles>
    <__roles>content_image</__roles>
    <__roles>subtitle_h3</__roles>
    <__roles>paragraph--1</__roles>
    <__roles>composite-video</__roles>
    <__roles>paragraph--2</__roles>
    <__roles>paragraph--3</__roles>
    <__roles>content_image--1</__roles>
    <__roles>article_related</__roles>
    <__roles>cite_block</__roles>
    <__roles>content_generic</__roles>
    <__roles>details_author_name</__roles>
    <__roles>details_author_location</__roles>
    <__roles>details_author_description</__roles>
    <__roles>details_author_facebook</__roles>
    <__roles>details_author_twitter</__roles>
    <__roles>details_author_instagram</__roles>
    <__roles>details_author_linkedin</__roles>
    ...
    <details_author_location>
        <content>Madrid</content>
        <authorId>2</authorId>
    </details_author_location>
    <details_author_name>
        <content>Miguel de Cervantes</content>
        <authorId>6</authorId>
    </details_author_name>
    <epigraph>
        <content>Maravilloso relato, de aventuras y locuras, dudo si es más famoso que exagerado.</content>
    </epigraph>
    ...

The previous XML snippet can be replaced by:

xml
  <modules>
    <autoroles />
    <title>New news</title>
    <epigraph>
      <content>Maravilloso relato, de aventuras y locuras, dudo si es más famoso que exagerado.</content>
    </epigraph>
    <details_author_name>
      <content>Miguel de Cervantes</content>
      <authorId>6</authorId>
    </details_author_name>
    <details_author_location>
        <content>Madrid</content>
        <authorId>2</authorId>
    </details_author_location>
    ...

Bulk import/update ​

There is a special bulk operation that can be used to mass import several entities, even if they are not the same type of entity, simply concatenate one after the other and they will be imported.

Then, with this input:

xml
<?xml version="1.0"?>
<item>
  <item>
    <data>
      <slug>new_user</slug>
      <firstName>User</firstName>
      ...
    </data>
    <type>user</type>
  </item>
  <item>
    <data>
      <name>622b48a46cd7d.jpeg</name>
      <title>nomnomnomnomnom</title>
      <description>622b48a46cd7d.jpeg</description>
      ...
    </data>
    <type>image</type>
  </item>
  <item>
    <data>
      <template>default</template>
      <uuid>xxxx1647346272-621646995299</uuid>
      <category>default-cat</category>
      <title>Noticia 0123</title>
    </data>
    <modules>
      <__roles>main_image</__roles>
      <__roles>supratitle</__roles>
      <__roles>title</__roles>
      ...
      <supratitle>
        <content>Supra title por ejemplo</content>
      </supratitle>
      <title>
        <content>Noticia 0123</content>
      </title>
    </modules>
    <type>article</type>
  </item>
</item>

Running this command:

shell
curl \
  -s \
  -X POST \
  -H "Authorization: Bearer $IAPI_TOKEN" \
  -H "Content-Type: application/xml" \
  -d @bulk.xml \
  "$IAPI_SERVER/bulk"

We get the output:

xml
<?xml version="1.0"?>
<item>
  <success>1</success>
  <item key="0">
    <type>user</type>
    <id>129</id>
  </item>
  <item key="1">
    <type>image</type>
    <id>1202</id>
  </item>
  <item key="2">
    <type>article</type>
    <id>3590</id>
  </item>
</item>

If no item.id is provided, then an import operation will be performed, if item.id is provided, then an update operation will be performed.

Known entities and structures ​

Articles ​

  • item
    • type, fixed to article.
    • data
      • uuid, external identifier.
      • template, default, opinion, recipes, ...
      • category, the category's slug.
      • title, informative title, it is not the published title.
      • slug, full slug without extension suffix nor platform suffix (i.e. category/20220124/1234/simple-title)
      • pageSlug, the name part on slug, if is sent, will be replaced on slug.
      • status, set published to publish the article.
      • publishedAt, you can set a future date for schedule publishing (format RFC3339).
      • firstPublishedAt, the first published date (format RFC3339).
    • modules, specific metadata structure (export examples to see how to import them).
    • platformModules, specific metadata structure for specific platforms (export examples to see how to import them). If you send this node, modules will be ignored for indicated platforms.
    • settings, specific metadata structure (export examples to see how to import them).
    • seo, specific metadata structure (export examples to see how to import them).
    • tags, tag list
      • tag, tag's id or tag's slug (must exists)
    • authors, author list
      • authorId, author id

Categories ​

  • item
    • type, fixed to category.
    • data
      • id, internal category id.
      • slug
      • title
      • parent, parent category id or null if it's root.
      • active, boolean
      • description
      • type, template type for wich articles can select this category.
      • template, template type for this category.
    • settings, specific metadata structure (export examples to see how to import them).

Images ​

  • item
    • type, fixed to image.
    • data
      • uuid, external identifier.
      • name, internal image name.
      • title, public title.
      • description, public description.
      • alt, public alternative text.
      • base64file, binary image data base64 encoded, equivalent to $ base64 < image.webp | perl -n -e 'chomp;print'

Videos ​

  • item
    • type, fixed to video.
    • data
      • uuid, unique identifier.
      • slug, video slug.
      • name, internal video name.
      • title, public title.
      • description, public description.
      • provider, video provider (if any, e.g. jwplatform)
      • providerId, video id on the provider system.
      • thumbnail, video thumbnail image ref (id or automapped), WARN one image can be assigned only to one video (the thumbnail is an unique key on videos)
      • base64file, binary video data base64 encoded

Users ​

  • item
    • id, internal user id.
    • type, fixed to user.
    • data
      • slug, unique user identifier.
      • location, any string.
      • signature, public author signature.
      • firstName
      • fullName
      • username, unique user alias.
      • email
      • plainPassword, used only first time to set one password.
      • avatar, user avatar image ref (id or automapped)
      • instagram, specific Instagram information
      • linkedin
      • contact, email contact
      • emailCanonical
      • lastName
      • columnist, boolean
      • hasPage, boolean, this author has their own public page
      • enabled, boolean, account enabled/disabled
      • roles, list
        • role, granted role
      • groups, list
        • group, granted role group (joined to)

Tags ​

  • item
    • slug, tag slug
    • active, bool
    • title, tag title
    • createdAt, creation date
    • updatedAt, last updated time

RedirectUrls ​

  • item
    • sourceUrl, string required, for every request, the request.uri will be compared with this sourceUrl field.
    • destinationUrl, string required, if any match, the request will be redirected to this destinationUrl url.
    • internal, boolean optional, true if the redirection must be done inside the server, false if a redirection code (see httpCode) must be sent to the client.
    • httpCode, string optional, a valid redirection HTTP code (default used 307).

EsiTags ​

  • item
    • tags, list
      • tag, esi tag to purge

Current available operations ​

text
DELETE /api/importer/tags/{id}                                  # deactivate the given tag by `id`
DELETE /api/importer/tags/slug/{slug}                           # deactivate the given tag by `slug`
DELETE /api/importer/tags/title/{title}                         # deactivate the given tag by `title`
DELETE /api/importer/redirecturls/{id}                          # delete one redirection by `id`
DELETE /api/importer/users/{id}                                 # delete one user by `id`
GET /api/importer/articles/{id}                                 # given one Xalok internal article `id` retrieve the article details
GET /api/importer/articles/mapping                              # return first 1000 articles with id >= 1
GET /api/importer/articles/mapping/{fromId}                     # return first 1000 articles with id >= fromId
GET /api/importer/articles/mapping/{fromId}/{maxResults}        # return first maxResults articles with id >= fromId, should be maxResults <= 100000
GET /api/importer/articles/mapping/{fromId}/{maxResults}/full   # return first maxResults articles with id >= fromId with extended information          
GET /api/importer/articles/uuid/{uuid}                          # given one `uuid` retrieve the article details
GET /api/importer/boards/{slug}                                 # given one board `slug` retrieve the board details
GET /api/importer/categories                                    # retrieve all categories
GET /api/importer/categories/{id}                               # given one Xalok internal category `id` retrieve the category details
GET /api/importer/images/uuid/{uuid}                            # given one Xalok internal image `uuid` retrieve the image details
GET /api/importer/images/{id}                                   # given one Xalok internal image `id` retrieve the image details
GET /api/importer/redirecturls                                  # retrieve all redirections
GET /api/importer/redirecturls/{id}                             # given one Xalok internal redirection `id` retrieve the redirection details
GET /api/importer/tags                                          # retrieve all tags
GET /api/importer/tags/{id}                                     # given one Xalok internal tag `id` retrieve the tag details
GET /api/importer/users                                         # retrieve the details of all users
GET /api/importer/users/{id}                                    # given one Xalok internal user `id` retrieve the user details
GET /api/importer/users/slug/{slug}                             # retrieve the user details by the user slug
GET /api/importer/users/name/{name}                             # retrieve the user details by the canonical user name
GET /api/importer/users/email/{email}                           # retrieve the user details by the canonical user email
GET /api/importer/version                                       # retrieve the current server API version
GET /api/importer/videos/{id}                                   # given one Xalok internal video `id` retrieve the video details
POST /api/importer/articles                                     # create a new one article reading the POST data with same format as exported
POST /api/importer/articles/uuid                                # search all articles matching with POSTed uuids
POST /api/importer/boards                                       # upsert one board where the key is the `slug` field
POST /api/importer/bulk                                         # digest many items of many types
POST /api/importer/categories                                   # create a new one category reading the POST data with same format as exported
POST /api/importer/esi/purge                                    # purge a list of esi tags
POST /api/importer/images                                       # create a new one image reading the POST data with same format as exported
POST /api/importer/redirecturls                                 # create a new one redirection reading the POST data with same format as exported
POST /api/importer/tags                                         # create a new one tag reading the POST data with same format as exported
POST /api/importer/users                                        # create a new one user reading the POST data with same format as exported
POST /api/importer/videos                                       # create a new one video reading the POST data with same format as exported
PUT /api/importer/articles                                      # update an existing article reading the PUT data with same format as exported
                                                                # (as the PK, item.id will be used or item.data.uuid)
PUT /api/importer/tags                                          # update an existing tag reading the PUT data with same format as exported
                                                                # (as the PK, item.id will)
PUT /api/importer/categories                                    # update an existing category reading the PUT data with same format as exported
                                                                # (as the PK, item.id will)
PUT /api/importer/images                                        # update an existing image reading the PUT data with same format as exported
                                                                # (as the PK, item.id will)
PUT /api/importer/redirecturls                                  # update an existing redirection reading the PUT data with same format as exported
                                                                # (as the PK, item.id will)
PUT /api/importer/videos                                        # update an existing videos reading the PUT data with same format as exported
                                                                # (as the PK, item.id will)
PUT /api/importer/tags/merge/{oldtitle}/{newtitle}              # move all relations with the `oldtitle` tag to the `newtitle` tag, 
                                                                # if `newtitle` does not exists, rename the `oldtitle` to the `newtitle` 
PUT /api/importer/users                                         # update an existing user reading the PUT data with same format as exported
                                                                # (as the PK, item.id will be used or item.data.slug)
                                                                # WARNING the slug will be enumerated if already exists!

/api/importer/articles/uuid ​

Bulk search by UUID operation:

shell
$ curl \
  -X POST \
  -ks \
  -H "Authorization: Bearer $IAPI_TOKEN" \
  -H "Content-Type: application/xml" \
  -d '<item>
        <uuid>ffc38e21-c041-43b1-bb35-8279d21c147e</uuid>
        <uuid>c7355dc6-e8ec-4aa9-b28b-8cfa25a30bea</uuid>
        <uuid>557bf9d2-aafc-41f3-ab69-23cba592fd1f</uuid>
        ...
    </item>' \
  "$IAPI_SERVER/articles/uuid" \

where resulting output contains the internal id at item.item@key:

xml
<?xml version="1.0"?>
<item>
  <item key="1446">
    <data>
      <template>default</template>
      <uuid>557bf9d2-aafc-41f3-ab69-23cba592fd1f</uuid>
      <category>extra-cat-13</category>
      <title>T&#xED;tulo DESKTOP</title>
      <status>published</status>
      <publishedAt>2022-06-02T15:44:54+02:00</publishedAt>
      <slug>extra-cat-13/tokamak/1446/titulo-desktop</slug>
      <pageSlug>titulo-desktop</pageSlug>
    </data>
    <modules>
      <facebook>
        <__roles>main_image</__roles>
        <__roles>sponsor</__roles>
        <__roles>title</__roles>
  ...