GSM 03.38, Unicode and Concatenated SMS: How to Calculate Segments Before Sending

What segment calculation solves

In SMS, counting visible characters is not enough. The number of segments depends on the text that is actually encoded, the selected alphabet and, when the content exceeds one segment, the concatenation mechanism. A visible character may consume one or two positions in GSM 7-bit, and a character outside the GSM 03.38 tables may require an alternative strategy, such as explicit encoding, conversion to UCS2, substitution, transformation or rejection, depending on the platform configuration.

The difference is operational. It affects volume estimates, cost control, the capacity reserved for a campaign and the recipient experience. It can also change after a template has been approved if a variable, signature, link or compliance text introduces different characters or exceeds a length threshold.

The useful rule is to always calculate based on the final rendered message, not an abstract template or a generic character counter.

• Keep the exact text that will be sent, including resolved variables.
• Determine the encoding before estimating segments.
• Count septets for GSM 7-bit, not only displayable characters.
• Count 16-bit UCS2 units when the text is handled as UCS2.
• Apply the capacity that corresponds to the configured concatenation method.

The two relevant encodings: GSM 03.38 and UCS2

For practical SMS planning, the two most relevant encodings are GSM 7-bit, based on the GSM 03.38 alphabet, and UCS2. GSM 7-bit can pack up to 160 septets into the 140 octets available for user data when there is no UDH. UCS2 uses 16-bit units within a repertoire limited to the Basic Multilingual Plane and allows up to 70 UCS2 units in a single-part message.

GSM 7-bit is not a visual subset of Unicode selected by appearance. It is a defined repertoire of basic characters and an extension table. If every character in the final text belongs to the applicable tables, the message can be calculated in GSM septets. If a character outside that repertoire appears, an alternative strategy is required. Many platforms encode it or convert it to UCS2; others may reject it, substitute it or require explicit encoding.

UCS2 is not equivalent to Unicode or UTF-16. UCS2 is a 16-bit repertoire and encoding limited to the Basic Multilingual Plane. Some implementations commercially referred to as “UCS2” may accept UTF-16 or apply their own transformations. Therefore, especially for characters outside the Basic Multilingual Plane, validate how the implementation and messaging provider measure, transform and segment the content.

• GSM 7-bit without UDH: up to 160 septets.
• UCS2 without UDH: up to 70 16-bit UCS2 units.
• Do not choose the encoding based on the campaign language: validate it against the final text.
• Do not assume an editor’s Unicode support is equivalent to GSM 7-bit compatibility.
• Validate the handling of characters outside the Basic Multilingual Plane in your specific implementation.

Practical per-segment limits and the effect of concatenation

A single-part message can carry up to 160 septets in GSM 7-bit or up to 70 16-bit UCS2 units. When content requires several segments, each part needs information so that the handset can identify the set and its order. That information is usually carried through a User Data Header, or UDH, and reduces the payload available for text.

With a concatenation IE using an 8-bit reference, the UDH occupies 6 octets. Of the 140-octet TP-UD limit, 134 octets remain for content. As a technical calculation, this corresponds to 153 GSM 7-bit septets per multipart segment or 67 UCS2 units per multipart segment. With a 16-bit reference, the UDH occupies one additional octet: practical capacity becomes 152 GSM 7-bit septets or 66 UCS2 units per multipart segment.

In GSM 7-bit with UDH, the calculation must account for septet padding and alignment according to TP-UDHI. The values 153 and 152 are common capacities derived from the indicated user-data limits and concatenation structures. The final implementation must measure the constructed TP-UD, especially when other IEs are added to the UDH. The actual method on a route may not match these assumptions; confirm the exact segmentation, splitting and billable counting method with your SMSC, route or provider before establishing it as a contractual or billing rule.

• GSM 7-bit, single-part message: 160 septets.
• UCS2, single-part message: 70 16-bit UCS2 units.
• Concatenated GSM 7-bit with an 8-bit reference: 153 septets per multipart segment.
• Concatenated UCS2 with an 8-bit reference: 67 UCS2 units per multipart segment.
• Concatenated GSM 7-bit with a 16-bit reference: 152 septets per multipart segment.
• Concatenated UCS2 with a 16-bit reference: 66 UCS2 units per multipart segment.
• Other IEs in the UDH may reduce available capacity further.

How to calculate segments in an auditable way

For GSM 7-bit, classify every character in the final text as belonging to the basic table or the extension table. Each basic-table character consumes one septet. Each extension-table character consumes two septets because it is encoded using an escape followed by the extension code.

The formula is: GSM septets = basic-table characters + 2 × extension-table characters. If the result fits within 160 septets, it is a single-part message. If it exceeds that limit, divide the total by the applicable concatenated capacity and round up. For an 8-bit reference, the usual planning capacity is 153 septets per multipart segment; for a 16-bit reference, it is 152. In GSM 7-bit with UDH, confirm the result from the constructed TP-UD and adjust capacity if other IEs are present in the UDH.

For UCS2, if the text consists of characters representable in one 16-bit UCS2 unit, count UCS2 units. Divide by 70 when there is no concatenation or by 67 when an 8-bit concatenation UDH is used, rounding up. These results represent UCS2 units, not universal Unicode characters. If your technical stack accepts UTF-16, characters outside the Basic Multilingual Plane or applies its own transformations, do not reuse this calculation without checking how those characters are measured, transformed and segmented.

• GSM 7-bit formula: basic characters + 2 × extended characters.
• Single-part GSM segments: 1 if septets are 160 or fewer.
• Concatenated GSM segments: round up total septets divided by 153 or 152, according to the UDH applied and after validating the constructed TP-UD.
• Single-part UCS2 segments: round up UCS2 units divided by 70.
• Concatenated UCS2 segments with an 8-bit reference: round up UCS2 units divided by 67.
• Adjust capacity if the UDH contains other IEs or the route uses another segmentation method.

GSM 03.38 extended characters: one visible character can occupy two septets

The GSM 7-bit extension table allows certain symbols to be retained without switching to UCS2, but they are not free in terms of capacity. These include €, ^, {, }, [, ], ~ and the backslash. Although each appears as a single character, it requires an escape sequence and consumes two septets.

This detail matters near the limits. A text with 158 visible characters does not necessarily fit in one GSM 7-bit SMS. If it contains three extension characters, its consumption would be 161 septets even if all other characters belong to the basic table. In that case, the text moves to concatenation.

Septet calculation for these symbols is determined by the GSM 03.38 table. Operationally, validate that the implementation and route apply the expected encoding. An operational counter should show at least three values: visible characters, number of extended characters and total GSM septets. Showing only visual length can lead to incorrect approvals.

• Common extended characters: €, ^, {, }, [, ], ~ and backslash.
• Each GSM extended character consumes two septets.
• Calculate septets according to the GSM 03.38 table and validate the route implementation.
• Pay particular attention to prices, amounts, codes, technical expressions and URLs containing symbols.
• Do not automatically substitute characters without an approved policy: substitution may change the content’s meaning.

What commonly triggers Unicode and how to prevent unexpected changes

Typographic quotation marks, emojis, non-Latin alphabets and many symbols copied from word processors, spreadsheets, design tools or content management systems may not be part of GSM 7-bit. This should not be decided by intuition: the validator must compare every content code point against the GSM basic table and extension table.

Straight quotation marks and typographic quotation marks look similar, but they do not necessarily have the same representation. The same applies to dashes, special spaces, registered marks, icons and stylistic variants of letters or numbers. The risk is not only applying a strategy other than GSM 7-bit, such as conversion to UCS2: a later change can increase the number of segments and alter the capacity forecast.

The strongest preventive measure is to normalize text according to a documented policy, retain both the original and normalized versions, and recalculate encoding and segments after every dynamic substitution. When a character outside GSM 7-bit is detected, the policy should define whether explicit encoding is used, the content is converted to UCS2, substituted, transformed or rejected.

• Detect characters outside GSM 7-bit before sending.
• Review content pasted from editing tools and template libraries.
• Validate variable values with the same engine used to validate the fixed message body.
• Keep original and normalized text so differences can be explained.
• Require new approval if the final text changes encoding or segment range.

Calculation examples

Example 1, OTP: “Your access code is 482913.” If all characters belong to the GSM basic table, the message is calculated in GSM septets and remains well within the 160 available in a single-part message. The operational recommendation is to run the calculation after inserting the actual code and any prefix or signature added in production.

Example 2, transactional notification: “Order 8742 confirmed. Total: 18€.” This example assumes that, apart from the € symbol, the remaining characters used belong to the applicable GSM basic repertoire. The € symbol belongs to the GSM extension table. The message can remain in GSM 7-bit, but € consumes two septets. For a short text, this does not change the number of parts, but it must be reflected in the counter and audit trail.

Example 3, consented promotional message: if an approved campaign includes a typographic quotation mark or an emoji, the text requires an alternative strategy to GSM 7-bit. Depending on the configuration, it may be converted to UCS2, require explicit encoding, be substituted, transformed or rejected. If UCS2 is used, single-part message capacity drops to 70 UCS2 units and multipart-segment capacity, with an 8-bit reference, drops to 67 under those UDH assumptions. Before changing the content, the owner must decide whether to retain the character, adopt an approved GSM-compatible alternative or redesign the text to control the number of parts.

In all three cases, estimates must not use a generic template length. They must include URLs, brand names, variables, regulatory text, STOP/HELP where applicable, and any text added by the sender system.

• OTP: calculate after inserting the final code.
• Transactional message with €: if the rest of the text belongs to the applicable GSM basic repertoire, remain in GSM 7-bit, but add one extra septet compared with a basic character.
• Consented promotional message with an emoji or typographic quotation marks: apply the configured strategy for characters outside GSM 7-bit.
• Do not use text examples as a substitute for validating every rendered message.

Operational planning and approval table

An operational table turns the technical calculation into a repeatable control. It should be recorded by template and version, and also whenever a variable can materially change the encoding or number of segments. These fields do not need to be treated as regulatory requirements; they are recommended controls for reviewing a decision and reproducing the send.

The expected-segments field should state the technical assumption used. For example, it is not enough to record “2 segments”: it is useful to record “2 GSM 7-bit, concatenation UDH with 8-bit reference, 153 septets per multipart segment.” This prevents another integration from applying a different method without detection.

In SMPP, SAR describes a concatenated message through parameters for the set reference, total number of segments and the position of each fragment. The SMPP specification does not generally guarantee that the SMSC will automatically segment the message or encapsulate the content in UDH: that behavior depends on the SMSC and the route’s technical agreement. In GSM networks, SMPP v3.4 indicates that concatenation SAR TLVs must not be combined with a concatenation UDH already encoded in short_message.

• Template identifier and version.
• Original text and text after normalization.
• Example of final rendered text with variables resolved.
• Expected encoding: GSM 7-bit or UCS2.
• Basic characters, extended characters and total septets, where applicable.
• UCS2 units, where applicable.
• Expected segments and the capacity threshold used.
• UDH type or SAR mechanism selected for SMPP concatenation, where applicable. In GSM networks, do not combine concatenation SAR TLVs with a concatenation UDH encoded in short_message. If a UDH is encoded in user data, enable the UDHI indicator in esm_class in accordance with SMPP v3.4.