9.2. Syntax Qualifier

  • Qualifier specifies what to find.

  • a - Exact

  • a|b - Alternative

  • [abc] - Enumeration

  • [a-z] - Range

9.2.1. SetUp

>>> import re

9.2.2. Exact

  • a - Exact

>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'

Regular expressions allows to find exact matches:

>>> re.findall(r'a', TEXT)
['a', 'a', 'a', 'a', 'a']

Note, that regular expressions are case sensitive (unless re.IGNORECASE flag is present. More information in Syntax Flags):

>>> re.findall(r'A', TEXT)
['A']

Note, that regular expressions are used to search in text, therefore in case of searching for a number it will return a strings with numbers in it:

>>> re.findall(r'1', TEXT)
['1']

Python re.findall() function will return empty list if none match was found:

>>> re.findall(r'x', TEXT)
[]

9.2.3. Exact Alternate

  • a|b - letter a or b (also works with expressions)

>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'

Alternative allows to search for two or more possible matches:

>>> re.findall(r'a|b', TEXT)
['a', 'a', 'a', 'a', 'a']

It can find more than two matches:

>>> re.findall(r'a|b|c|d', TEXT)
['a', 'a', 'a', 'd', 'd', 'a', 'a']
>>>
>>> re.findall(r'1|2|3', TEXT)
['3', '2', '3', '1', '3']

It will work for both numbers, characters or any other object:

>>> re.findall(r'a|b|c|d|1|2|3', TEXT)
['a', 'a', '3', 'a', 'd', 'd', 'a', '2', '3', 'a', '1', '3']

Examples:

>>> re.findall(r'a|e', TEXT)
['a', 'a', 'e', 'e', 'a', 'e', 'a', 'a']
>>>
>>> re.findall(r'a|e|i|o|u|y', TEXT)
['a', 'a', 'e', 'y', 'o', 'e', 'a', 'e', 'o', 'a', 'o', 'o', 'a']

9.2.4. Enumeration

  • [abc] - letter a or b or c

>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'

Enumerations provide compact and more readable syntax for longer alternatives:

>>> re.findall(r'[abcd]', TEXT)
['a', 'a', 'a', 'd', 'd', 'a', 'a']
>>>
>>> re.findall(r'[123]', TEXT)
['3', '2', '3', '1', '3']

It will work for both numbers, characters or any other object:

>>> re.findall(r'[abcd123]', TEXT)
['a', 'a', '3', 'a', 'd', 'd', 'a', '2', '3', 'a', '1', '3']

Examples:

>>> re.findall(r'[a-z]', TEXT)  
['a', 'r', 'k', 'a', 't', 'n', 'e', 'y', 'o', 'f', 'r', 'e', 's', 'l',
 'a', 'n', 'd', 'e', 'd', 'o', 'n', 'a', 'r', 's', 'o', 'n', 'o', 'v',
 't', 'h', 'a', 't', 'p', 'm']
>>>
>>> re.findall(r'[az-]', TEXT)  
['a', 'a', 'a', 'a', 'a']
>>> re.findall(r'[A-z]', TEXT)  
['M', 'a', 'r', 'k', 'W', 'a', 't', 'n', 'e', 'y', 'o', 'f', 'A', 'r',
 'e', 's', 'l', 'a', 'n', 'd', 'e', 'd', 'o', 'n', 'M', 'a', 'r', 's',
 'o', 'n', 'N', 'o', 'v', 't', 'h', 'a', 't', 'p', 'm']
>>>
>>> re.findall(r'[a-Z]', TEXT)
Traceback (most recent call last):
re.error: bad character range a-Z at position 1
>>>
>>> re.findall(r'[z-a]', TEXT)
Traceback (most recent call last):
re.error: bad character range z-a at position 1

Use Cases:

>>> re.findall(r'[aeiouy]', TEXT)
['a', 'a', 'e', 'y', 'o', 'e', 'a', 'e', 'o', 'a', 'o', 'o', 'a']
>>>
>>> re.findall(r'a|e|i|o|u|y', TEXT)
['a', 'a', 'e', 'y', 'o', 'e', 'a', 'e', 'o', 'a', 'o', 'o', 'a']

9.2.5. Range

  • [a-z] - any lowercase ASCII letter from a to z

  • [A-Z] - any uppercase ASCII letter from A to Z

  • [0-9] - any digit from 0 to 9

  • [a-zA-Z] - any ASCII letter from: a to z or from A to Z

  • [A-z] - any ASCII letter from: a to z or from A to Z

  • [a-zA-Z0-9] - any ASCII letter from a to z or from A to Z or digit from 0 to 9

>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'

Ranges provide even more readable and convenient way os specifying particular characters to match. It is very useful to define ranges of numbers or letters this way:

>>> re.findall(r'[a-z]', TEXT)  
['a', 'r', 'k', 'a', 't', 'n', 'e', 'y', 'o', 'f', 'r', 'e', 's', 'l',
 'a', 'n', 'd', 'e', 'd', 'o', 'n', 'a', 'r', 's', 'o', 'n', 'o', 'v',
 't', 'h', 'a', 't', 'p', 'm']
>>>
>>> re.findall(r'[A-Z]', TEXT)
['M', 'W', 'A', 'M', 'N']
>>>
>>> re.findall(r'[0-9]', TEXT)
['3', '7', '2', '0', '3', '5', '1', '3', '7']

Note, that regular expressions are case sensitive (unless re.IGNORECASE flag is present. More information in Syntax Flags). You can also join ranges to create even broader matches:

>>> re.findall(r'[a-zA-Z]', TEXT)  
['M', 'a', 'r', 'k', 'W', 'a', 't', 'n', 'e', 'y', 'o', 'f', 'A', 'r',
 'e', 's', 'l', 'a', 'n', 'd', 'e', 'd', 'o', 'n', 'M', 'a', 'r', 's',
 'o', 'n', 'N', 'o', 'v', 't', 'h', 'a', 't', 'p', 'm']
>>>
>>> re.findall(r'[a-zA-Z0-9]', TEXT)  
['M', 'a', 'r', 'k', 'W', 'a', 't', 'n', 'e', 'y', 'o', 'f', 'A', 'r',
 'e', 's', '3', 'l', 'a', 'n', 'd', 'e', 'd', 'o', 'n', 'M', 'a', 'r',
 's', 'o', 'n', 'N', 'o', 'v', '7', 't', 'h', '2', '0', '3', '5', 'a',
 't', '1', '3', '7', 'p', 'm']

Ranges are ordered in ASCII table order (more information in Locale Encoding). Because uppercase letters are before lowercase letters (has lower indexes), you can define range from Z-a, but the opposite is not true:

>>> re.findall(r'[Z-a]', TEXT)
['a', 'a', 'a', 'a', 'a']
>>> re.findall(r'[a-Z]', TEXT)
Traceback (most recent call last):
re.error: bad character range a-Z at position 1

The last example can work in some other languages due to the different implementation of the algorithm or PCRE standard. More information in Syntax Extensions.

Mind that ranges not necessarily need to be from a-z. It could be any alphabetic or numeric range:

>>> re.findall(r'[3-7]', TEXT)
['3', '7', '3', '5', '3', '7']
>>>
>>> re.findall(r'[C-Y]', TEXT)
['M', 'W', 'M', 'N']
>>>
>>> re.findall(r'[3-7C-Y]', TEXT)
['M', 'W', '3', 'M', 'N', '7', '3', '5', '3', '7']

9.2.6. Joining

  • [abc]|[123] - Enumeration alternative - letter a, b or c or digit 1, 2 3

  • [a-z]|[0-9] - Range alternative - any lowercase ASCII letter from a to z or digit from 0 to 9

>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'

Alternative enumerations syntax is as follows:

>>> re.findall(r'[abc]|[123]', TEXT)
['a', 'a', '3', 'a', 'a', '2', '3', 'a', '1', '3']

The effect is identical to:

>>> re.findall(r'[abc123]', TEXT)
['a', 'a', '3', 'a', 'a', '2', '3', 'a', '1', '3']

You can define alternative ranges to find:

>>> re.findall(r'[A-Z]|[0-9]', TEXT)
['M', 'W', 'A', '3', 'M', 'N', '7', '2', '0', '3', '5', '1', '3', '7']

The effect is identical to:

>>> re.findall(r'[A-Z0-9]', TEXT)
['M', 'W', 'A', '3', 'M', 'N', '7', '2', '0', '3', '5', '1', '3', '7']

9.2.7. Examples

  • [d-m] - any lowercase letter from d to m

  • [3-7] - any digit from 3 to 7

  • [xz2] - x or z or 2

  • [d-mK-P3-8] - any lowercase letter from d to m or uppercase letter from K to P or digit from 3 to 8

  • x|z|2 - x or z or 2

  • d|x - d or x

  • [d-k]|[ABC]|[3-8] - any lowercase letter from d to k or uppercase A,`B` or C or digit from 3 to 8

9.2.8. Use Case - 0x01

>>> import re
>>> TEXT = 'Mark Watney of Ares 3 landed on Mars on: Nov 7th, 2035 at 1:37 pm'

Note, the nd in word landed:

>>> re.findall(r'st|nd|rd|th', TEXT)
['nd', 'th']
>>> re.findall(r'[st|nd|rd|th]', TEXT)
['r', 't', 'n', 'r', 's', 'n', 'd', 'd', 'n', 'r', 's', 'n', 't', 'h', 't']
>>> re.findall(r'[stndrdth]', TEXT)
['r', 't', 'n', 'r', 's', 'n', 'd', 'd', 'n', 'r', 's', 'n', 't', 'h', 't']

9.2.9. Assignments

Code 9.9. Solution
"""
* Assignment: RE Syntax Qualifier
* Complexity: easy
* Lines of code: 3 lines
* Time: 3 min

English:
    1. Use regular expressions find in text:
        a. all digits
        b. all uppercase letters
        c. all lowercase letters
    2. Run doctests - all must succeed

Polish:
    1. Użyj wyrażeń regularnych wyszukiwania w tekście
        a. wszystkie cyfry
        b. wszystkie duże litery
        c. wszystkie małe litery
    2. Uruchom doctesty - wszystkie muszą się powieść

Hint:
    * `re.findall()`

References:
    [1] Authors: Wikipedia contributors
        Title: Apollo 11
        Publisher: Wikipedia
        Year: 2019
        Retrieved: 2019-12-14
        URL: https://en.wikipedia.org/wiki/Apollo_11

Tests:
    >>> import sys; sys.tracebacklimit = 0
    >>> from pprint import pprint

    >>> pprint(result_a, compact=True, width=72)
    ['1', '1', '2', '0', '1', '9', '6', '9', '2', '0', '1', '7', '6', '3',
     '9', '2', '1', '1', '9', '6', '9', '0', '2', '5', '6', '1', '5', '1',
     '9', '2', '3', '1', '4', '7', '5', '2', '1', '5', '2', '1', '3', '6']

    >>> pprint(result_b, compact=True, width=72)
    ['A', 'A', 'M', 'C', 'C', 'D', 'R', 'N', 'A', 'L', 'M', 'P', 'B', 'A',
     'A', 'L', 'M', 'L', 'M', 'E', 'J', 'U', 'T', 'C', 'A', 'E', 'V', 'A',
     'M', 'E', 'V', 'A', 'J', 'U', 'T', 'C', 'A', 'T', 'T', 'B', 'A', 'A',
     'E', 'M', 'C', 'C', 'M', 'P', 'C', 'M', 'C', 'M', 'C', 'M', 'C']

    >>> pprint(result_c, compact=True, width=72)
    ['p', 'o', 'l', 'l', 'o', 'w', 'a', 's', 't', 'h', 'e', 'm', 'e', 'r',
     'i', 'c', 'a', 'n', 's', 'p', 'a', 'c', 'e', 'f', 'l', 'i', 'g', 'h',
     't', 't', 'h', 'a', 't', 'f', 'i', 'r', 's', 't', 'l', 'a', 'n', 'd',
     'e', 'd', 'h', 'u', 'm', 'a', 'n', 's', 'o', 'n', 't', 'h', 'e', 'o',
     'o', 'n', 'o', 'm', 'm', 'a', 'n', 'd', 'e', 'r', 'e', 'i', 'l', 'r',
     'm', 's', 't', 'r', 'o', 'n', 'g', 'a', 'n', 'd', 'l', 'u', 'n', 'a',
     'r', 'm', 'o', 'd', 'u', 'l', 'e', 'p', 'i', 'l', 'o', 't', 'u', 'z',
     'z', 'l', 'd', 'r', 'i', 'n', 'l', 'a', 'n', 'd', 'e', 'd', 't', 'h',
     'e', 'p', 'o', 'l', 'l', 'o', 'u', 'n', 'a', 'r', 'o', 'd', 'u', 'l',
     'e', 'a', 'g', 'l', 'e', 'o', 'n', 'u', 'l', 'y', 't', 'h', 'a', 't',
     'a', 'n', 'd', 'r', 'm', 's', 't', 'r', 'o', 'n', 'g', 'b', 'e', 'c',
     'a', 'm', 'e', 't', 'h', 'e', 'f', 'i', 'r', 's', 't', 'p', 'e', 'r',
     's', 'o', 'n', 't', 'o', 's', 't', 'e', 'p', 'o', 'n', 't', 'o', 't',
     'h', 'e', 'o', 'o', 'n', 's', 's', 'u', 'r', 'f', 'a', 'c', 'e', 'h',
     'o', 'u', 'r', 's', 'm', 'i', 'n', 'u', 't', 'e', 's', 'l', 'a', 't',
     'e', 'r', 'o', 'n', 'u', 'l', 'y', 's', 't', 'a', 't', 'l', 'd', 'r',
     'i', 'n', 'j', 'o', 'i', 'n', 'e', 'd', 'h', 'i', 'm', 'm', 'i', 'n',
     'u', 't', 'e', 's', 'l', 'a', 't', 'e', 'r', 'h', 'e', 'y', 's', 'p',
     'e', 'n', 't', 'h', 'o', 'u', 'r', 's', 'm', 'i', 'n', 'u', 't', 'e',
     's', 'e', 'x', 'p', 'l', 'o', 'r', 'i', 'n', 'g', 't', 'h', 'e', 's',
     'i', 't', 'e', 't', 'h', 'e', 'y', 'h', 'a', 'd', 'n', 'a', 'm', 'e',
     'd', 'r', 'a', 'n', 'q', 'u', 'i', 'l', 'i', 't', 'y', 'a', 's', 'e',
     'u', 'p', 'o', 'n', 'l', 'a', 'n', 'd', 'i', 'n', 'g', 'r', 'm', 's',
     't', 'r', 'o', 'n', 'g', 'a', 'n', 'd', 'l', 'd', 'r', 'i', 'n', 'c',
     'o', 'l', 'l', 'e', 'c', 't', 'e', 'd', 'p', 'o', 'u', 'n', 'd', 's',
     'k', 'g', 'o', 'f', 'l', 'u', 'n', 'a', 'r', 'm', 'a', 't', 'e', 'r',
     'i', 'a', 'l', 't', 'o', 'b', 'r', 'i', 'n', 'g', 'b', 'a', 'c', 'k',
     't', 'o', 'a', 'r', 't', 'h', 'a', 's', 'p', 'i', 'l', 'o', 't', 'i',
     'c', 'h', 'a', 'e', 'l', 'o', 'l', 'l', 'i', 'n', 's', 'f', 'l', 'e',
     'w', 't', 'h', 'e', 'o', 'm', 'm', 'a', 'n', 'd', 'o', 'd', 'u', 'l',
     'e', 'o', 'l', 'u', 'm', 'b', 'i', 'a', 'i', 'n', 'l', 'u', 'n', 'a',
     'r', 'o', 'r', 'b', 'i', 't', 'a', 'n', 'd', 'w', 'e', 'r', 'e', 'o',
     'n', 't', 'h', 'e', 'o', 'o', 'n', 's', 's', 'u', 'r', 'f', 'a', 'c',
     'e', 'f', 'o', 'r', 'h', 'o', 'u', 'r', 's', 'm', 'i', 'n', 'u', 't',
     'e', 's', 'b', 'e', 'f', 'o', 'r', 'e', 'l', 'i', 'f', 't', 'i', 'n',
     'g', 'o', 'f', 'f', 't', 'o', 'r', 'e', 'j', 'o', 'i', 'n', 'o', 'l',
     'u', 'm', 'b', 'i', 'a']

"""

import re

TEXT = """Apollo 11 was the American spaceflight that first landed
humans on the Moon. Commander (CDR) Neil Armstrong and lunar module
pilot (LMP) Buzz Aldrin landed the Apollo Lunar Module (LM) Eagle on
July 20th, 1969 at 20:17 UTC, and Armstrong became the first person
to step (EVA) onto the Moon's surface (EVA) 6 hours 39 minutes later,
on July 21st, 1969 at 02:56:15 UTC. Aldrin joined him 19 minutes later.
They spent 2 hours 31 minutes exploring the site they had named Tranquility
Base upon landing. Armstrong and Aldrin collected 47.5 pounds (21.5 kg)
of lunar material to bring back to Earth as pilot Michael Collins (CMP)
flew the Command Module (CM) Columbia in lunar orbit, and were on the
Moon's surface for 21 hours 36 minutes before lifting off to rejoin
Columbia."""

# Find all digits in text
# Example: '1', '1', '2', '0', '1', '9', '6', ...
# type: list[str]
result_a = ...

# Find all uppercase letters in text
# Example: 'A', 'A', 'M', 'C', 'C', 'D', 'R', ...
# type: list[str]
result_b = ...

# Find all lowercase letters in text
# Example: 'p', 'o', 'l', 'l', 'o', ...
# type: list[str]
result_c = ...

Code 9.10. Solution
"""
* Assignment: RE Syntax Qualifier
* Complexity: easy
* Lines of code: 1 lines
* Time: 3 min

English:
    1. Use regular expressions find in text:
        a. all digits and uppercase letters
    2. Run doctests - all must succeed

Polish:
    1. Użyj wyrażeń regularnych wyszukiwania w tekście:
        a. wszystkie cyfry i duże litery
    2. Uruchom doctesty - wszystkie muszą się powieść

Hint:
    * `re.findall()`

References:
    [1] Authors: Wikipedia contributors
        Title: Apollo 11
        Publisher: Wikipedia
        Year: 2019
        Retrieved: 2019-12-14
        URL: https://en.wikipedia.org/wiki/Apollo_11

Tests:
    >>> import sys; sys.tracebacklimit = 0
    >>> from pprint import pprint

    >>> pprint(result, compact=True, width=72)
    ['A', '1', '1', 'A', 'M', 'C', 'C', 'D', 'R', 'N', 'A', 'L', 'M', 'P',
     'B', 'A', 'A', 'L', 'M', 'L', 'M', 'E', 'J', '2', '0', '1', '9', '6',
     '9', '2', '0', '1', '7', 'U', 'T', 'C', 'A', 'E', 'V', 'A', 'M', 'E',
     'V', 'A', '6', '3', '9', 'J', '2', '1', '1', '9', '6', '9', '0', '2',
     '5', '6', '1', '5', 'U', 'T', 'C', 'A', '1', '9', 'T', '2', '3', '1',
     'T', 'B', 'A', 'A', '4', '7', '5', '2', '1', '5', 'E', 'M', 'C', 'C',
     'M', 'P', 'C', 'M', 'C', 'M', 'C', 'M', '2', '1', '3', '6', 'C']
"""

import re

TEXT = """Apollo 11 was the American spaceflight that first landed
humans on the Moon. Commander (CDR) Neil Armstrong and lunar module
pilot (LMP) Buzz Aldrin landed the Apollo Lunar Module (LM) Eagle on
July 20th, 1969 at 20:17 UTC, and Armstrong became the first person
to step (EVA) onto the Moon's surface (EVA) 6 hours 39 minutes later,
on July 21st, 1969 at 02:56:15 UTC. Aldrin joined him 19 minutes later.
They spent 2 hours 31 minutes exploring the site they had named Tranquility
Base upon landing. Armstrong and Aldrin collected 47.5 pounds (21.5 kg)
of lunar material to bring back to Earth as pilot Michael Collins (CMP)
flew the Command Module (CM) Columbia in lunar orbit, and were on the
Moon's surface for 21 hours 36 minutes before lifting off to rejoin
Columbia."""

# Find all digits and uppercase letters in text
# Example: 'A', '1', '1', 'A', 'M', 'C', 'C', 'D', 'R', ...
# type: list[str]
result = ...