Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Language and Locale
Language is a term that just about everybody is familiar with; it is a grouping of sounds and symbols that are organized into combinations and patterns that enable people to communicate. A locale, on the other hand, is a specific international market where a target user is working. A locale encompasses many of the issues discussed earlier when I defined localization: the conventions, culture, language, and preferences of typical users in the environment. This includes issues that are crucial to applications, such as number, currency, date/time formats, and date/time separators.
Usually, when you are discussing issues in i18N, it makes more sense to refer to locales than to languages. The reason for this is twofold:
-
The language alone is usually not a good indicator for items defined in a locale. For example, in both America and Great Britain, the language spoken is English. However, in America the built-in short date format is mm/dd/yy, so January 4, 2000, would be represented as 01/04/00. In Great Britain, the dd/mm/yy format is preferred, which means that people there would expect the same date to be represented as 04/01/00. There is obviously a lot of room for confusion within an application if you display such information or your users type such information in the wrong format.
-
In many cases, a language is spoken in very different ways between two different regions. One example of this is the difference between Brazilian Portuguese and Iberian Portuguese (as spoken in Portugal). For example, the following phrase "data that is stored in a Microsoft Jet or Microsoft SQL Server database" might be translated to "dados queest[td]ao armazenados em uma base de dados do Microsoft Jet ou do Microsoft SQL Server" for Brazilian Portuguese, but into "esses dados s[td]ao armazenados numa base de dados do Microsoft Jet ou do Microsoft SQL Server" in Iberian Portuguese. It might look like a minor difference, but it's one that people who speak the language will notice.
Throughout the book, special care will be used to be very specific about referring to language and locale as appropriate.
Locale ID (LCID)
In an effort to define locales in ways that enable users to choose a single setting rather than having to individually set every preference, Microsoft has defined a large database of information that is intended to be specific to every covered language, country, and sort combination. Not every possible locale is covered; in fact, almost without exception, languages are supported with an LCID only if there is a country or at least a region that speaks the language. (Therefore, you will probably never see an LCID for Coptic or Latin, and you will certainly never see an LCID for Klingon, even though a Klingon extension to the Unicode standard has been proposed and has even been included in some Unix implementations!)
LCIDs are made up of four parts:
-
PRIMARYLANGID—A primary language ID, which defines the language
-
SUBLANGID—A sublanguage identifier, which defines the country or region
-
SORTID—A sort identifier, which defines the way that the language is sorted
-
SORTVERSION—A sort version, which is not currently used but is defined so that future versions of Windows can use alternate versions of a sort if this is ever required
The layout of the LCID in memory can be seen in Figure 10.
){kind=link}
Figure 10 The layout in memory of a 32-bit LCID value. The currently used sort version uses a small part of the reserved area.
LCIDs will be used at many points throughout this book. C and C++ developers have been given many helpful macros to parse these numbers and obtain the relevant portions of them. Although these macros are not available in Visual Basic, I do provide wrapper functions that provide the same functionality (the main difference is that in C, the macros would be inserted into every instance in which they are used by a preprocesser, whereas in VB they are actual functions that will exist even in a compiled project).
The information represented by an LCID (usually pronounced el-sid) is used heavily by Visual Basic, by COM, and by the operating system itself, and its settings fundamentally affect the processing of information on the machine. LCIDs will be heavily discussed throughout the book. The specific parts of LCIDs are further discussed in the following sections.
Primary Language Identifier (PRIMARYLANGID)
As I mentioned, the primary language identifier represents the language itself. A good example is English, which is spoken in many countries but always has the same PRIMARYLANGID, LANG_ENGLISH. Some countries have multiple languages that are spoken; therefore, LANG_TAMIL and LANG_BENGALI both exist to represent two different languages spoken in India.
Sublanguage Identifier (SUBLANGID)
The SUBLANGID represents the country or region in which the language is spoken. This is obviously needed for cases such as French spoken in Belgium (represented by SUBLANG_FRENCH_BELGIAN) as compared to the French spoken in Canada (represented by SUBLANG_FRENCH_CANADIAN). The distinction between sublanguages is important just for word usage issues within a language, even before taking into account the other issues that make up a locale. This makes the SUBLANGID a very important component.
Language Identifier (LANGID)
The LANGID, or language identifier, is simply a combination of the PRIMARYLANGID and SUBLANGID. As the next section discussing the sort identifier indicates, in most cases, the LANGID and the LCID will be the same value because most languages do not have special sorts.
Sort Identifier (SORTID)
At first glance, PRIMARYLANGID and SUBLANGID seem to be able to cover everything, but, in the end, that proves to be a bad assumption. The differences in the way that Hungarian is sorted under normal circumstances versus in technical documentation, and the way that Georgian is sorted in the modern versus the traditional use of the language, are indeed important cultural factors that can be key to properly displaying information in an appropriate order. In most cases, SORT_DEFAULT (which is 0) is used, thus making the LANGID and the LCID identical.
Sort Version (SORTVERSION)
Recent versions of the Platform SDK have added macros to handle a sort version. It is currently unused and even the documentation in the SDK does not discuss what its intended future use might be. (It simply states that this parameter is reserved and must always be 0.) I would suspect that this part of the LCID is present so that future versions of sorts that are almost identical but have small variations can be represented in ways that will allow Windows to optimize sorting operations.
Because no sort versions are currently defined, they are not really discussed much in the rest of the book, although the macros that manipulate them have been converted to functions just as the other LCID macros have.
It is worth noting that there are specific times in which a SORTID or even a SORTVERSION would probably have been a good way to handle a specific language difference, but it was not used. The best example of this is the difference between the Traditional Sort Spanish LANGID (1034) and the Modern Sort Spanish LANGID (3082). There are no differences in any of the locale settings or characters in either language. The difference between the two is that the way particular characters are sorted has been changed. Specifically, in the traditional sort order, the letter combination ch is a unique compression in the sort between c and d; in the modern sort, it's considered as c followed by h. Similarly, in the traditional sort, the ll compression is a unique letter in the sort between l and m; in the modern sort, it is considered as l followed by l. This would have made Modern Sort Spanish an ideal candidate to be defined as LANGID 1034 with a new SORTID; unfortunately, this was not done, and Modern Spanish has its own LCID. So, instead of an LCID of 66570, Modern Spanish has an LCID of 3082. In fact, the reason that Modern Spanish (whose name, incidentally, was changed to Spanish [International] in Windows 2000) was not implemented with a new sort is that the SORTID component of LCIDs did not exist until more than a year later.
Worldwide EXE
The concept of the worldwide EXE is simple enough; it means that your application's code has one version for all languages. At first glance, the best way to handle other locales is to take the product that you have written and give it to a programmer from the other locale. For example, you would take your U.S. English program and give it to a Japanese developer, who could take care of changing the strings in VB forms, the code, and everything else to make sure that it is appropriate to the Japanese market. There are three major problems with this approach:
-
All the localized versions of the software will be available much later than the original version. Because you would be handing off your finished application, you have to wait until it is done or you risk having to fix bugs in multiple places.
-
It is much more expensive to have extra programmers do the work. As you add on languages, you are essentially paying for one programmer per language. This can become prohibitively expensive.
-
Your product is going to be either a very hard-to-maintain single version or multiple versions. It is hard to say whether it is worse to have to maintain multiple versions of a product or maintain a product with code such as this:
Select Case g_language
Case ENGLISH_VER
' English-specific code here
Case JAPANESE_VER
' Japanese-specific code here
Case GERMAN_VER
' German-specific code here
End Select
However, this approach will be very difficult in terms of maintaining the product, fixing bugs, or shipping future versions.
The solution is to have a single binary that contains the code and to isolate materials that need to be localized in another form (such as a database or resource files).
The concept is referred to as worldwide EXE even though, in truth, the worldwide component of your product can easily be many files that are EXEs, DLLs, ActiveX controls, and other files. In all cases, the idea is to separate from your core program anything that will change from language to language and also design your core program to be able to handle any language.
In the real world of software development, it is often slightly more complicated. Such a product cycle, which has happened for Visual Basic itself (in this case of the Package and Deploy [P&D] Wizard), goes something like:
-
The product ships its source version (U.S. English).
-
The localized versions of the product are shipped at a later date. The worldwide EXE approach was taken, but the later ship date means that testing happened later and that some bugs were fixed after the original ship date.
-
An "internationally friendly" version of the P&D Wizard designed to enable VB developers to ship their localized applications was released at a later date and made available on the Web (at http://msdn.microsoft.com/vbasic/downloads/ipdk.asp, which, incidentally, has non-English links as well; an excellent step for a feature that will attract global interest).
The important difference between this type of product cycle and the original one discussed is that these changes are now considered part of the core code. Any and all bug fixes made along the way appear in all subsequent versions. In essence, what the VB team did was further globalize its product. Many bugs found by local testers were considered core bugs as a part of the worldwide EXE.
Input Method Editor (IME)
Everyone knows what a keyboard is, and keyboards are very useful when you need to type in text for most languages. When you are dealing with the Asian languages (also known as the Chinese/Japanese/Korean, or CJK, languages), a simple keyboard will not do the trick because there is no keyboard that has thousands of keys on it.
To solve this problem, Microsoft uses IMEs (input method editors) to allow for the input of such alphabets. These editors (some of which can appear in the list of input locales, as in Figure 11, and which often look much like the Japanese IME 2000 in Figure 12) provide the ability to input grouping information that approximates a character and then lets you choose the right character as needed.
){kind=link}
){kind=link}
Figure 11 If you install a lot of keyboards on Windows 2000, you may notice that many of the Asian languages look different than they do in other locales; these different graphics represent different IMEs.
Figure 12 Selecting the Japanese IME 2000 on a Windows 2000 machine will give you an interface in which you can handle the large number of Japanese Hiragana, Katakana, and Kanji characters that you may want to enter.
The IME is run by the Input Method Manager (IMM) API and is usually a system component and thus not one you will have to worry about very much.
Where We Just Were…
I have now introduced the subjects that will be covered in depth for the rest of the book, and I have defined many of the terms that I will be using, in case you either did not know any of the terms or perhaps might be using different ones than I am using. This should help direct your further reading and make sure that you will find this book to be a useful tool in the journey to a Visual Basic application that enables you to take advantage of the global marketplace.