strutils
- Text manipulation¶
So much practical programming involves string manipulation, which
Python readily accommodates. Still, there are dozens of basic and
common capabilities missing from the standard library, several of them
provided by strutils
.
-
boltons.strutils.
camel2under
(camel_string)[source]¶ Converts a camelcased string to underscores. Useful for turning a class name into a function name.
>>> camel2under('BasicParseTest') 'basic_parse_test'
-
boltons.strutils.
under2camel
(under_string)[source]¶ Converts an underscored string to camelcased. Useful for turning a function name into a class name.
>>> under2camel('complex_tokenizer') 'ComplexTokenizer'
-
boltons.strutils.
slugify
(text, delim='_', lower=True, ascii=False)[source]¶ A basic function that turns text full of scary characters (i.e., punctuation and whitespace), into a relatively safe lowercased string separated only by the delimiter specified by delim, which defaults to
_
.The ascii convenience flag will
asciify()
the slug if you require ascii-only slugs.>>> slugify('First post! Hi!!!!~1 ') 'first_post_hi_1'
>>> slugify("Kurt Gödel's pretty cool.", ascii=True) == b'kurt_goedel_s_pretty_cool' True
-
boltons.strutils.
split_punct_ws
(text)[source]¶ While
str.split()
will split on whitespace,split_punct_ws()
will split on punctuation and whitespace. This used internally byslugify()
, above.>>> split_punct_ws('First post! Hi!!!!~1 ') ['First', 'post', 'Hi', '1']
-
boltons.strutils.
unit_len
(sized_iterable, unit_noun='item')[source]¶ Returns a plain-English description of an iterable’s
len()
, conditionally pluralized withcardinalize()
, detailed below.>>> print(unit_len(range(10), 'number')) 10 numbers >>> print(unit_len('aeiou', 'vowel')) 5 vowels >>> print(unit_len([], 'worry')) No worries
-
boltons.strutils.
ordinalize
(number, ext_only=False)[source]¶ Turns number into its cardinal form, i.e., 1st, 2nd, 3rd, 4th, etc. If the last character isn’t a digit, it returns the string value unchanged.
Parameters: >>> print(ordinalize(1)) 1st >>> print(ordinalize(3694839230)) 3694839230th >>> print(ordinalize('hi')) hi >>> print(ordinalize(1515)) 1515th
-
boltons.strutils.
cardinalize
(unit_noun, count)[source]¶ Conditionally pluralizes a singular word unit_noun if count is not one, preserving case when possible.
>>> vowels = 'aeiou' >>> print(len(vowels), cardinalize('vowel', len(vowels))) 5 vowels >>> print(3, cardinalize('Wish', 3)) 3 Wishes
-
boltons.strutils.
pluralize
(word)[source]¶ Semi-intelligently converts an English word from singular form to plural, preserving case pattern.
>>> pluralize('friend') 'friends' >>> pluralize('enemy') 'enemies' >>> pluralize('Sheep') 'Sheep'
-
boltons.strutils.
singularize
(word)[source]¶ Semi-intelligently converts an English plural word to its singular form, preserving case pattern.
>>> singularize('chances') 'chance' >>> singularize('Activities') 'Activity' >>> singularize('Glasses') 'Glass' >>> singularize('FEET') 'FOOT'
-
boltons.strutils.
asciify
(text, ignore=False)[source]¶ Converts a unicode or bytestring, text, into a bytestring with just ascii characters. Performs basic deaccenting for all you Europhiles out there.
Also, a gentle reminder that this is a utility, primarily meant for slugification. Whenever possible, make your application work with unicode, not against it.
Parameters: >>> asciify('Beyoncé') == b'Beyonce' True
-
boltons.strutils.
is_ascii
(text)[source]¶ Check if a unicode or bytestring, text, is composed of ascii characters only. Raises
ValueError
if argument is not text.Parameters: text (str or unicode) – The string to be checked. >>> is_ascii('Beyoncé') False >>> is_ascii('Beyonce') True
-
boltons.strutils.
is_uuid
(obj, version=4)[source]¶ Check the argument is either a valid UUID object or string.
Parameters: >>> is_uuid('e682ccca-5a4c-4ef2-9711-73f9ad1e15ea') True >>> is_uuid('0221f0d9-d4b9-11e5-a478-10ddb1c2feb9') False >>> is_uuid('0221f0d9-d4b9-11e5-a478-10ddb1c2feb9', version=1) True
-
boltons.strutils.
html2text
(html)[source]¶ Strips tags from HTML text, returning markup-free text. Also, does a best effort replacement of entities like “ ”
>>> r = html2text(u'<a href="#">Test &<em>(Δημώ)</em></a>') >>> r == u'Test &(Δημώ)' True
-
boltons.strutils.
strip_ansi
(text)[source]¶ Strips ANSI escape codes from text. Useful for the occasional time when a log or redirected output accidentally captures console color codes and the like.
>>> strip_ansi('[0m[1;36mart[46;34m') 'art'
Supports unicode, str, bytes and bytearray content as input. Returns the same type as the input.
There’s a lot of ANSI art available for testing on sixteencolors.net. This function does not interpret or render ANSI art, but you can do so with ansi2img or escapes.js.
-
boltons.strutils.
bytes2human
(nbytes, ndigits=0)[source]¶ Turns an integer value of nbytes into a human readable format. Set ndigits to control how many digits after the decimal point should be shown (default
0
).>>> bytes2human(128991) '126K' >>> bytes2human(100001221) '95M' >>> bytes2human(0, 2) '0.00B'
Finds and returns all hashtags in a string, with the hashmark removed. Supports full-width hashmarks for Asian languages and does not false-positive on URL anchors.
>>> find_hashtags('#atag http://asite/#ananchor') ['atag']
find_hashtags
also works with unicode hashtags.
-
boltons.strutils.
a10n
(string)[source]¶ That thing where “internationalization” becomes “i18n”, what’s it called? Abbreviation? Oh wait, no:
a10n
. (It’s actually a form of numeronym.)>>> a10n('abbreviation') 'a10n' >>> a10n('internationalization') 'i18n' >>> a10n('') ''
-
boltons.strutils.
gzip_bytes
(bytestring, level=6)[source]¶ Turn some bytes into some compressed bytes.
>>> len(gzip_bytes(b'a' * 10000)) 46
Parameters: Note that all levels of gzip are pretty fast these days, though it’s not really a competitor in compression, at any level.
-
boltons.strutils.
gunzip_bytes
(bytestring)[source]¶ The
gzip
module is great if you have a file or file-like object, but what if you just have bytes. StringIO is one possibility, but it’s often faster, easier, and simpler to just use this one-liner. Use this tried-and-true utility function to decompress gzip from bytes.>>> gunzip_bytes(_EMPTY_GZIP_BYTES) == b'' True >>> gunzip_bytes(_NON_EMPTY_GZIP_BYTES).rstrip() == b'bytesahoy!' True
-
boltons.strutils.
iter_splitlines
(text)[source]¶ Like
str.splitlines()
, but returns an iterator of lines instead of a list. Also similar tofile.next()
, as that also lazily reads and yields lines from a file.This function works with a variety of line endings, but as always, be careful when mixing line endings within a file.
>>> list(iter_splitlines('\nhi\nbye\n')) ['', 'hi', 'bye', ''] >>> list(iter_splitlines('\r\nhi\rbye\r\n')) ['', 'hi', 'bye', ''] >>> list(iter_splitlines('')) []
-
boltons.strutils.
indent
(text, margin, newline='\n', key=<class 'bool'>)[source]¶ The missing counterpart to the built-in
textwrap.dedent()
.Parameters:
-
boltons.strutils.
escape_shell_args
(args, sep=' ', style=None)[source]¶ Returns an escaped version of each string in args, according to style.
Parameters: - args (list) – A list of arguments to escape and join together
- sep (str) – The separator used to join the escaped arguments.
- style (str) – The style of escaping to use. Can be one of
cmd
orsh
, geared toward Windows and Linux/BSD/etc., respectively. If style isNone
, then it is picked according to the system platform.
See
args2cmd()
andargs2sh()
for details and example output for each style.
-
boltons.strutils.
args2cmd
(args, sep=' ')[source]¶ Return a shell-escaped string version of args, separated by sep, using the same rules as the Microsoft C runtime.
>>> print(args2cmd(['aa', '[bb]', "cc'cc", 'dd"dd'])) aa [bb] cc'cc dd\"dd
As you can see, escaping is through backslashing and not quoting, and double quotes are the only special character. See the comment in the code for more details. Based on internal code from the
subprocess
module.
-
boltons.strutils.
args2sh
(args, sep=' ')[source]¶ Return a shell-escaped string version of args, separated by sep, based on the rules of sh, bash, and other shells in the Linux/BSD/MacOS ecosystem.
>>> print(args2sh(['aa', '[bb]', "cc'cc", 'dd"dd'])) aa '[bb]' 'cc'"'"'cc' 'dd"dd'
As you can see, arguments with no special characters are not escaped, arguments with special characters are quoted with single quotes, and single quotes themselves are quoted with double quotes. Double quotes are handled like any other special character.
Based on code from the
pipes
/shlex
modules. Also note thatshlex
andargparse
have functions to split and parse strings escaped in this manner.
-
boltons.strutils.
parse_int_list
(range_string, delim=', ', range_delim='-')[source]¶ Returns a sorted list of positive integers based on range_string. Reverse of
format_int_list()
.Parameters: - range_string (str) – String of comma separated positive integers or ranges (e.g. ‘1,2,4-6,8’). Typical of a custom page range string used in printer dialogs.
- delim (char) – Defaults to ‘,’. Separates integers and contiguous ranges of integers.
- range_delim (char) – Defaults to ‘-’. Indicates a contiguous range of integers.
>>> parse_int_list('1,3,5-8,10-11,15') [1, 3, 5, 6, 7, 8, 10, 11, 15]
-
boltons.strutils.
format_int_list
(int_list, delim=', ', range_delim='-', delim_space=False)[source]¶ Returns a sorted range string from a list of positive integers (int_list). Contiguous ranges of integers are collapsed to min and max values. Reverse of
parse_int_list()
.Parameters: - int_list (list) – List of positive integers to be converted into a range string (e.g. [1,2,4,5,6,8]).
- delim (char) – Defaults to ‘,’. Separates integers and contiguous ranges of integers.
- range_delim (char) – Defaults to ‘-’. Indicates a contiguous range of integers.
- delim_space (bool) – Defaults to
False
. IfTrue
, adds a space after all delim characters.
>>> format_int_list([1,3,5,6,7,8,10,11,15]) '1,3,5-8,10-11,15'
-
class
boltons.strutils.
MultiReplace
(sub_map, **kwargs)[source]¶ MultiReplace is a tool for doing multiple find/replace actions in one pass.
Given a mapping of values to be replaced it allows for all of the matching values to be replaced in a single pass which can save a lot of performance on very large strings. In addition to simple replace, it also allows for replacing based on regular expressions.
Keyword Arguments:
Parameters: Dictionary Usage:
from boltons import stringutils s = stringutils.MultiReplace({ 'foo': 'zoo', 'cat': 'hat', 'bat': 'kraken' }) new = s.sub('The foo bar cat ate a bat') new == 'The zoo bar hat ate a kraken'
Iterable Usage:
from boltons import stringutils s = stringutils.MultiReplace([ ('foo', 'zoo'), ('cat', 'hat'), ('bat', 'kraken)' ]) new = s.sub('The foo bar cat ate a bat') new == 'The zoo bar hat ate a kraken'
The constructor can be passed a dictionary or other mapping as well as an iterable of tuples. If given an iterable, the substitution will be run in the order the replacement values are specified in the iterable. This is also true if it is given an OrderedDict. If given a dictionary then the order will be non-deterministic:
>>> 'foo bar baz'.replace('foo', 'baz').replace('baz', 'bar') 'bar bar bar' >>> m = MultiReplace({'foo': 'baz', 'baz': 'bar'}) >>> m.sub('foo bar baz') 'baz bar bar'
This is because the order of replacement can matter if you’re inserting something that might be replaced by a later substitution. Pay attention and if you need to rely on order then consider using a list of tuples instead of a dictionary.
-
boltons.strutils.
multi_replace
(text, sub_map, **kwargs)[source]¶ Shortcut function to invoke MultiReplace in a single call.
Example Usage:
from boltons.stringutils import multi_replace new = multi_replace( 'The foo bar cat ate a bat', {'foo': 'zoo', 'cat': 'hat', 'bat': 'kraken'} ) new == 'The zoo bar hat ate a kraken'
-
boltons.strutils.
unwrap_text
(text, ending='\n\n')[source]¶ Unwrap text, the natural complement to
textwrap.wrap()
.>>> text = "Short \n lines \nwrapped\nsmall.\n\nAnother\nparagraph." >>> unwrap_text(text) 'Short lines wrapped small.\n\nAnother paragraph.'
Parameters: - text – A string to unwrap.
- ending (str) – The string to join all unwrapped paragraphs
by. Pass
None
to get the list. Defaults to ‘nn’ for compatibility with Markdown and RST.