Localization within the Backend API

The translation process in the backend app uses the machine translation engine provided by the Crowdin API. Crowdin's Machine Translation Engine (MTS) is a feature that provides machine translations for various languages. It is a convenient and efficient way to translate large amounts of content quickly and cost-effectively. Crowdin's MTS uses advanced machine learning algorithms to analyze and translate source content. It's important to note that machine translations may not always be as accurate as human translations, so it's important to thoroughly review and edit the translations generated by MTS to ensure they are of high quality. However, it can still be a useful tool to speed up the translation process, especially for content that is more straightforward or technical in nature, in the case of the backend API we will be using it to translate only short strings of text from English to Arabic.

Crowdin also integrates support for other translation engines like, Google Translate, Microsoft Translator, Translate, DeepL Pro, Amazon Translate, and Watson (IBM) Translate engines. But for this project, we will only use the Crowdin MTS.

Here is a reference to the API documentation by Crowdin on how to use the MTS: https://developer.crowdin.com/api/v2/#operation/api.mts.getMany

Storing of Translated Texts

In our backend app, we use Mongoose to create schemas for our MongoDB collections. When we want to save translated fields for a document in a collection, we create additional fields in the schema to store the translated values.

Let's take the course collection as an example. Here is an example Mongoose schema for the course collection:

const courseSchema = new Schema({
    author: {
        type: String,
        required: true
    },
    title: {
        type: String,
        required: true
    },
    description: {
        type: String,
        required: true
    },
    title_tr: {
        type: String,
    },
    description_tr: {
        type: String,
    },
    videos: [{ type: Schema.Types.ObjectId, ref: "Video" }],
    enrolled_users: [{ type: Schema.Types.ObjectId, ref: "User" }],
    preview_image: { type: String, required: true },
    isAvailable: { type: Boolean, default: true }
}, options);

As you can see, the courseSchema includes a title field, a description field, as well as corresponding title_tr and description_tr fields for storing the translated versions of the title and description fields.

When we translate a course document, we use the translateDoc() function, which was described in the previous discussion, to retrieve the values of the title and description fields, and store their translated versions in the title_tr and description_tr fields, respectively.

This way, we can easily retrieve the original or translated values of the fields as needed.

Code Implementation for Translation

Now, regarding the implementation in the backend using the machine translation engine provided by the Crowdin API:

Specifically, it uses the Machine Translation Service (MTS) provided by Crowdin to automatically translate strings of text from English to Arabic. Here is the simple code snippet used within the backend API to translate the strings of text from English to Arabic:

async function translateStringsWithCrowdinAPI(str_arr) {
    const url = `https://api.crowdin.com/api/v2/mts/${CROWDIN_MTS_ID}/translations`
    const res = await axios.post(
        url,
        {
            "languageRecognitionProvider": "crowdin",
            "targetLanguageId": "ar",
            "sourceLanguageId": "en",
            "strings": str_arr,
        },
        auth)

    // Replace the strings with their translations
    const translated_strings = res.data.data.translations

    return translated_strings
}

The code snippet above is a function that takes an array of strings and returns an array of translated strings. It does this by sending a POST request to the Crowdin MTS endpoint with the strings to be translated and the target and source languages specified. Once the request is made, the function waits for the response from the Crowdin API, which includes the translated strings.

To carry out translation using the Machine Translation Engine, you'll need to select any translation engine of your choice, each translation engine has an id attached to it, this id will be needed when making requests to carry out translation. In our case the CROWDIN_MTS_ID holds the id of the translation engine we chose.

You can make a request to the Crowdin API to get the list of available translation engines and their MTS ID, for more info visit here

Here is the code snippet used within the backend API to translate the strings of text from English to Arabic:

async function translateDoc(document) {
    let data = document;

    const string_indexes = {};
    const strings_to_translate = [];

    // Fields that we want to translate
    const translatable_fields = {
        'title': 'title',
        'description': 'description',
        'question': 'question',
        'correct_option': 'correct_option',
    }

    // Get the indexes of the strings to translate
    // and add the field index to the string_indexes object
    let string_index = 0;
    for (const field in data.toObject()) {
        if (translatable_fields[field]) {
            string_indexes[field] = string_index;
            string_index++;
            strings_to_translate.push(data[field]);
        }
    }

    // Use the Crowdin API to translate the strings
    const translated_strings = await translateStringsWithCrowdinAPI(strings_to_translate)

    // Replace the strings with their translations
    // Add the translated string to the data object
    for (const field in string_indexes) {
        data[field + '_tr'] = translated_strings[string_indexes[field]];
    }

    // If the document is a question, also translate the options and correct option
    if (data.type === "question") {
        data.options_tr = await translateStringsWithCrowdinAPI(data.options);

        const translated_correct_option = data.options_tr[data.correct_option];
        data.correct_option = translated_correct_option[0];
    }

    return data;
}

The translateDoc function takes a mongoose document as input and iterates through its fields to identify which ones are translatable. It then builds an array of strings to be translated and sends them to the translateStringsWithCrowdinAPI function.

Once the translated strings are returned, the translateDoc function replaces the original strings with their translated versions in the data object. If the document is a question, the options and correct option fields are also translated.

The use of the Crowdin API machine translation engine simplifies the translation process by automating the translation of strings, making it faster and more accurate than manual translation.