1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
|
<!DOCTYPE html>
<html lang="en" data-content_root="./">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Porting tips — Gentoo Python Guide documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=d1102ebc" />
<link rel="stylesheet" type="text/css" href="_static/basic.css?v=686e5160" />
<link rel="stylesheet" type="text/css" href="_static/alabaster.css?v=27fed22d" />
<script src="_static/documentation_options.js?v=5929fcd5"></script>
<script src="_static/doctools.js?v=9bcbadda"></script>
<script src="_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Migration guides" href="migration.html" />
<link rel="prev" title="Integration with build systems written in Python" href="buildsys.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="porting-tips">
<h1>Porting tips<a class="headerlink" href="#porting-tips" title="Link to this heading">¶</a></h1>
<p>This section highlights some of the known incompatible changes made
in Python that could break Python scripts and modules that used to work
in prior versions. The sections are split into retroactive changes made
to all Python releases, and information specific to every Python branch
(compared to the previous one).</p>
<p>This guide is by no means considered complete. If you can think
of other problems you’ve hit while porting your packages, please let me
know and I will update it.</p>
<section id="retroactive-changes">
<h2>Retroactive changes<a class="headerlink" href="#retroactive-changes" title="Link to this heading">¶</a></h2>
<section id="bpo43882-urlsplit-now-strips-lf-cr-and-ht-characters">
<h3><a class="reference external" href="https://bugs.python.org/issue43882">bpo43882</a>: urlsplit now strips LF, CR and HT characters<a class="headerlink" href="#bpo43882-urlsplit-now-strips-lf-cr-and-ht-characters" title="Link to this heading">¶</a></h3>
<p>Changed in: 2.7.18_p9, 3.6.13_p3, 3.7.10_p3, 3.8.9_p2, 3.9.4_p1</p>
<p>Historically, various <a class="reference external" href="https://docs.python.org/3/library/urllib.parse.html">urllib.parse</a> methods have passed special
characters such as LF, CR and HT through into the split URL components.
This could have resulted in various exploits if Python programs did not
validate the resulting components and used them verbatim.</p>
<p><a class="reference external" href="https://bugs.python.org/issue43882">bpo43882</a> attempted to address the issue by making <a class="reference external" href="https://docs.python.org/3/library/urllib.parse.html">urllib.parse</a> strip
the three aforementioned characters from the output of its functions.
This fixed one class of potential issues but at the same time opened
another can of worms. For example, URL validators that used to check
for dangerous special characters in the split URL components stopped
working correctly. In the best case, the URL were now sanitized instead
of being rejected. In the worst, the original unparsed URL with
dangerous characters started being passed through. See e.g. <a class="reference external" href="https://github.com/django/django/pull/14349">django
PR#14349</a> for an example of impact and a fix.</p>
<p>Behavior before:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">urlparse</span><span class="p">(</span><span class="s1">'https://example.com/bad</span><span class="se">\n</span><span class="s1">url'</span><span class="p">)</span>
<span class="go">ParseResult(scheme='https', netloc='example.com', path='/bad\nurl', params='', query='', fragment='')</span>
</pre></div>
</div>
<p>Behavior after:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">urlparse</span><span class="p">(</span><span class="s1">'https://example.com/bad</span><span class="se">\n</span><span class="s1">url'</span><span class="p">)</span>
<span class="go">ParseResult(scheme='https', netloc='example.com', path='/badurl', params='', query='', fragment='')</span>
</pre></div>
</div>
</section>
</section>
<section id="experimental-python-implementations">
<h2>Experimental Python implementations<a class="headerlink" href="#experimental-python-implementations" title="Link to this heading">¶</a></h2>
<section id="pypy">
<h3>PyPy<a class="headerlink" href="#pypy" title="Link to this heading">¶</a></h3>
<p>PyPy is using JIT to run Python code which can result in significant
performance improvements in some workflows, but it could also penalize
other programs.</p>
<p>In particular, calls to compiled extensions can penalize PyPy severely.
For this reason, it is generally recommended to skip building “speedup”
extensions for PyPy — in fact, upstream build systems frequently do that
automatically. A common approach is to combine checks for PyPy with
<code class="docutils literal notranslate"><span class="pre">native-extensions</span></code> USE flag:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_compile<span class="o">()</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span>!<span class="w"> </span>use<span class="w"> </span>native-extensions<span class="w"> </span><span class="o">||</span><span class="w"> </span><span class="o">[[</span><span class="w"> </span><span class="si">${</span><span class="nv">EPYTHON</span><span class="si">}</span><span class="w"> </span><span class="o">==</span><span class="w"> </span>pypy3<span class="w"> </span><span class="o">]]</span><span class="p">;</span><span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">MULTIDICT_NO_EXTENSIONS</span><span class="o">=</span><span class="m">1</span>
<span class="w"> </span><span class="k">fi</span>
<span class="w"> </span>distutils-r1_python_compile
<span class="o">}</span>
</pre></div>
</div>
<p>However, this does not imply that Python packages largely based
on C extensions should not be marked for PyPy3 compatibility.
For example, while packages such as Pandas can be less performant
under PyPy, they could be used as a part of larger program that overall
benefits from running under PyPy.</p>
</section>
<section id="freethreading-cpython-versions">
<span id="index-0"></span><h3>Freethreading CPython versions<a class="headerlink" href="#freethreading-cpython-versions" title="Link to this heading">¶</a></h3>
<p>CPython is used the so-called “global interpreter lock” to prevent
multiple threads from executing Python code simultaneously. It was
designed like this to simplify implementation, but at the cost of
limiting the possible performance gain from use of multiple threads.
Historically, only compiled extensions could temporarily disable it
to take advantage of threading.</p>
<p>Starting with CPython 3.13, a support for an experimental
“<a class="reference external" href="https://docs.python.org/3/howto/free-threading-python.html">freethreading</a>” mode has been added. If CPython is built in this
variant, it uses more fine-grained locking mechanisms to make it
possible to run multiple Python threads simultaneously. Note that this
mode can only improve performance of multithreaded applications,
and (at least at this point) penalizes single-threaded applications.</p>
<p>The freethreaded versions of CPython are not ABI-compatible with
the regular builds. Therefore, they are slotted as separate versions
with a <code class="docutils literal notranslate"><span class="pre">t</span></code> suffix, e.g. Python 3.13 freethreading can be found
as <code class="docutils literal notranslate"><span class="pre">dev-lang/python:3.13t</span></code>. They also need to be tested and added
to <code class="docutils literal notranslate"><span class="pre">PYTHON_COMPAT</span></code> separately, e.g. as <code class="docutils literal notranslate"><span class="pre">python3_13t</span></code>.</p>
<p>Note that compiled extensions need to declare support for freethreading
explicitly (see: <a class="reference external" href="https://docs.python.org/3/howto/free-threading-extensions.html">C API Extension Support for Free Threading</a>). While
extensions without explicit support can be compiled for freethreading
Python versions, importing them will cause GIL to be reenabled
and therefore defeat the purpose of freethreading build. To determine
whether a C extension supports freethreading mode, grep the code
for <code class="docutils literal notranslate"><span class="pre">Py_MOD_GIL_NOT_USED</span></code>. CPython will also verbosely warn upon
importing extensions without this support.</p>
<p>In general, do not add <code class="docutils literal notranslate"><span class="pre">python3_13t</span></code> to <code class="docutils literal notranslate"><span class="pre">PYTHON_COMPAT</span></code> in leaf
packages, unless they make use of multithreading and have real gain
from freethreaded versions — otherwise, it may actually be slower than
the regular variant.</p>
<p>For dependency packages, add <code class="docutils literal notranslate"><span class="pre">python3_13t</span></code> only after explicitly
testing that the package in question works. Do not add it
if the package in question installs extensions that do not support
freethreading. This would penalize the setup, prevent proper testing
and therefore defeat the purpose of separately specifying this target.
Preferably, wait until they do. For some common dependencies, adding it
may be acceptable, provided that the extensions are optional and that
they are not built for freethreading targets.</p>
</section>
</section>
<section id="python-3-13">
<h2>Python 3.13<a class="headerlink" href="#python-3-13" title="Link to this heading">¶</a></h2>
<p>See also: <a class="reference external" href="https://docs.python.org/3.13/whatsnew/3.13.html">what’s new in Python 3.13</a></p>
<section id="cgi-module-removal">
<span id="index-1"></span><h3>cgi module removal<a class="headerlink" href="#cgi-module-removal" title="Link to this heading">¶</a></h3>
<p>Python 3.13 removed the deprecated <a class="reference external" href="https://docs.python.org/3.12/library/cgi.html">cgi</a> module that provided a number
of utilities for CGI scripts. The standard library documentation
(as of Python 3.12) provides detailed information on replacing
the deprecated functions.</p>
<p>The said documentation recommends the <a class="reference external" href="https://pypi.org/project/multipart/">multipart</a> package
as a replacement for some of the functions. In its context,
the multipart vs. python-multipart packages section of <a class="reference internal" href="migration.html"><span class="doc">Migration guides</span></a>
should be consulted as well.</p>
</section>
<section id="docstring-dedenting">
<h3>Docstring dedenting<a class="headerlink" href="#docstring-dedenting" title="Link to this heading">¶</a></h3>
<p>Prior to Python 3.13, all whitespace in docstrings would be preserved
and exposed in the <code class="docutils literal notranslate"><span class="pre">__doc__</span></code> attribute. Starting with 3.13,
docstrings are instead stripped and dedented. For example, consider
the following docstring:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">frobnicate</span><span class="p">(</span><span class="n">thing</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""Frobnicate the thing</span>
<span class="sd"> Do some magic frobnication on the thing.</span>
<span class="sd"> Example::</span>
<span class="sd"> frobnicated_thing = frobnicate(thing)</span>
<span class="sd"> """</span>
</pre></div>
</div>
<p>In Python 3.12, all whitespace is preserved in <code class="docutils literal notranslate"><span class="pre">__doc__</span></code>, yielding:</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>Frobnicate the thing
Do some magic frobnication on the thing.
Example::
frobnicated_thing = frobnicate(thing)
</pre></div>
</div>
<p>Python 3.13 instead strips leading whitespace from the first line,
and the common amount of whitespace from the subsequent lines, yielding:</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>Frobnicate the thing
Do some magic frobnication on the thing.
Example::
frobnicated_thing = frobnicate(thing)
</pre></div>
</div>
<p>This can break some tests that rely on specific <code class="docutils literal notranslate"><span class="pre">__doc__</span></code> values.
To ensure consistent results, <a class="reference external" href="https://docs.python.org/3.13/library/inspect.html#inspect.cleandoc">inspect.cleandoc()</a> can be used
to perform the same operation in older Python versions, i.e.:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">assert</span> <span class="n">inspect</span><span class="o">.</span><span class="n">cleandoc</span><span class="p">(</span><span class="n">frobnicate</span><span class="o">.</span><span class="vm">__doc__</span><span class="p">)</span> <span class="o">==</span> <span class="n">expected_doc</span>
</pre></div>
</div>
</section>
</section>
<section id="python-3-12">
<h2>Python 3.12<a class="headerlink" href="#python-3-12" title="Link to this heading">¶</a></h2>
<p>See also: <a class="reference external" href="https://docs.python.org/3.12/whatsnew/3.12.html">what’s new in Python 3.12</a></p>
<section id="called-with-and-other-invalid-assertions-now-trigger-an-error">
<h3>.called_with (and other invalid assertions) now trigger an error<a class="headerlink" href="#called-with-and-other-invalid-assertions-now-trigger-an-error" title="Link to this heading">¶</a></h3>
<p>It is not uncommon for test suites to write invalid assertions such as:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">with</span> <span class="n">unittest</span><span class="o">.</span><span class="n">mock</span><span class="o">.</span><span class="n">patch</span><span class="p">(</span><span class="s2">"..."</span><span class="p">)</span> <span class="k">as</span> <span class="n">foo_mock</span><span class="p">:</span>
<span class="o">...</span>
<span class="k">assert</span> <span class="n">foo_mock</span><span class="o">.</span><span class="n">called_with</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
</pre></div>
</div>
<p>Prior to Python 3.12, such assertions would silently pass. Since
the <code class="docutils literal notranslate"><span class="pre">.called_with()</span></code> method does not exist, a <code class="docutils literal notranslate"><span class="pre">MagicMock</span></code> object
is returned and it evaluates to <code class="docutils literal notranslate"><span class="pre">True</span></code> in boolean context.</p>
<p>Starting with Python 3.12, an exception is raised instead:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="ne">AttributeError</span><span class="p">:</span> <span class="s1">'called_with'</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">a</span> <span class="n">valid</span> <span class="n">assertion</span><span class="o">.</span> <span class="n">Use</span> <span class="n">a</span> <span class="n">spec</span> <span class="k">for</span> <span class="n">the</span> <span class="n">mock</span> <span class="k">if</span> <span class="s1">'called_with'</span> <span class="ow">is</span> <span class="n">meant</span> <span class="n">to</span> <span class="n">be</span> <span class="n">an</span> <span class="n">attribute</span><span class="o">.</span>
</pre></div>
</div>
<p>The fix is to use the correct <code class="docutils literal notranslate"><span class="pre">.assert_called_with()</span></code> method
or similar:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">with</span> <span class="n">unittest</span><span class="o">.</span><span class="n">mock</span><span class="o">.</span><span class="n">patch</span><span class="p">(</span><span class="s2">"..."</span><span class="p">)</span> <span class="k">as</span> <span class="n">foo_mock</span><span class="p">:</span>
<span class="o">...</span>
<span class="n">foo_mock</span><span class="o">.</span><span class="n">assert_called_with</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
</pre></div>
</div>
<p>See the <a class="reference external" href="https://docs.python.org/3.12/library/unittest.mock.html">unittest.mock</a> documentation for the complete list of available
assertions.</p>
<p>Please note that since the original code did not actually test anything,
fixing the test case may reveal failed expectations.</p>
</section>
<section id="deprecated-test-method-alias-removal">
<h3>Deprecated test method alias removal<a class="headerlink" href="#deprecated-test-method-alias-removal" title="Link to this heading">¶</a></h3>
<p>Python 3.12 removes multiple deprecated test method aliases, such
as <code class="docutils literal notranslate"><span class="pre">assertEquals()</span></code> and <code class="docutils literal notranslate"><span class="pre">assertRegexpMatches()</span></code>. The documentation
provides <a class="reference external" href="https://docs.python.org/3.12/whatsnew/3.12.html#id3">a list of removed aliases and their modern replacements</a>.</p>
<p>It should be noted that all of the new methods are available since
Python 3.2 (and most even earlier), so the calls can be replaced without
worrying about backwards compatibility.</p>
<p>Most of the time, it should be possible to trivially <code class="docutils literal notranslate"><span class="pre">sed</span></code> the methods
in ebuild without having to carry a patch, e.g.:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>src_prepare<span class="o">()</span><span class="w"> </span><span class="o">{</span>
<span class="w"> </span><span class="c1"># https://github.com/byroot/pysrt/commit/93f52f6d4f70f4e18dc71deeaae0ec1e9100a50f</span>
<span class="w"> </span>sed<span class="w"> </span>-i<span class="w"> </span>-e<span class="w"> </span><span class="s1">'s:assertEquals:assertEqual:'</span><span class="w"> </span>tests/*.py<span class="w"> </span><span class="o">||</span><span class="w"> </span>die
<span class="w"> </span>distutils-r1_src_prepare
<span class="o">}</span>
</pre></div>
</div>
</section>
</section>
<section id="python-3-11">
<h2>Python 3.11<a class="headerlink" href="#python-3-11" title="Link to this heading">¶</a></h2>
<p>See also: <a class="reference external" href="https://docs.python.org/3.11/whatsnew/3.11.html">what’s new in Python 3.11</a></p>
<section id="generator-based-coroutine-removal-asyncio-coroutine">
<h3>Generator-based coroutine removal (asyncio.coroutine)<a class="headerlink" href="#generator-based-coroutine-removal-asyncio-coroutine" title="Link to this heading">¶</a></h3>
<p>Support for <a class="reference external" href="https://docs.python.org/3.10/library/asyncio-task.html#generator-based-coroutines">generator-based coroutines</a> has been deprecated since
Python 3.8, and is finally removed in 3.11. This usually results
in the following error:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="ne">AttributeError</span><span class="p">:</span> <span class="n">module</span> <span class="s1">'asyncio'</span> <span class="n">has</span> <span class="n">no</span> <span class="n">attribute</span> <span class="s1">'coroutine'</span>
</pre></div>
</div>
<p>The recommended solution is to use <a class="reference external" href="https://docs.python.org/3.10/library/asyncio-task.html#coroutines">PEP 492 coroutines</a>. They are
available since Python 3.5. This means replacing
the <code class="docutils literal notranslate"><span class="pre">@asyncio.coroutine</span></code> decorator with <code class="docutils literal notranslate"><span class="pre">async</span> <span class="pre">def</span></code> keyword,
and <code class="docutils literal notranslate"><span class="pre">yield</span> <span class="pre">from</span></code> with <code class="docutils literal notranslate"><span class="pre">await</span></code>.</p>
<p>For example, the following snippet:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="nd">@asyncio</span><span class="o">.</span><span class="n">coroutine</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">():</span>
<span class="k">yield from</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
</pre></div>
</div>
<p>would become:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">async</span> <span class="k">def</span> <span class="nf">foo</span><span class="p">():</span>
<span class="k">await</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
</pre></div>
</div>
</section>
<section id="inspect-getargspec-and-inspect-formatargspec-removal">
<h3>inspect.getargspec() and inspect.formatargspec() removal<a class="headerlink" href="#inspect-getargspec-and-inspect-formatargspec-removal" title="Link to this heading">¶</a></h3>
<p>The <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.getargspec">inspect.getargspec()</a> (deprecated since Python 3.0)
and <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.formatargspec">inspect.formatargspec()</a> (deprecated since Python 3.5) functions
are both removed in Python 3.11.</p>
<p>The <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.getargspec">inspect.getargspec()</a> function provides a legacy interface
to inspect the signature of callables. It is replaced
by the object-oriented <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.signature">inspect.signature()</a> API (available since
Python 3.3), or a mostly compatible <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.getfullargspec">inspect.getfullargspec()</a> function
(available since Python 3.0).</p>
<p>For example, a trivial function would yield the following results:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">p1</span><span class="p">,</span> <span class="n">p2</span><span class="p">,</span> <span class="o">/</span><span class="p">,</span> <span class="n">kp3</span><span class="p">,</span> <span class="n">kp4</span> <span class="o">=</span> <span class="mi">10</span><span class="p">,</span> <span class="n">kp5</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">pass</span>
<span class="gp">...</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">getargspec</span><span class="p">(</span><span class="n">foo</span><span class="p">)</span>
<span class="go">ArgSpec(args=['p1', 'p2', 'kp3', 'kp4', 'kp5'],</span>
<span class="go"> varargs='args',</span>
<span class="go"> keywords='kwargs',</span>
<span class="go"> defaults=(10, None))</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">getfullargspec</span><span class="p">(</span><span class="n">foo</span><span class="p">)</span>
<span class="go">FullArgSpec(args=['p1', 'p2', 'kp3', 'kp4', 'kp5'],</span>
<span class="go"> varargs='args',</span>
<span class="go"> varkw='kwargs',</span>
<span class="go"> defaults=(10, None),</span>
<span class="go"> kwonlyargs=[],</span>
<span class="go"> kwonlydefaults=None,</span>
<span class="go"> annotations={})</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">signature</span><span class="p">(</span><span class="n">foo</span><span class="p">)</span>
<span class="go"><Signature (p1, p2, /, kp3, kp4=10, kp5=None, *args, **kwargs)></span>
</pre></div>
</div>
<p>The named tuple returned by <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.getfullargspec">inspect.getfullargspec()</a> starts with
the same information, except that the key used to hold the name
of <code class="docutils literal notranslate"><span class="pre">**</span></code> parameter is <code class="docutils literal notranslate"><span class="pre">varkw</span></code> rather than <code class="docutils literal notranslate"><span class="pre">keywords</span></code>.
<a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.signature">inspect.signature()</a> returns a <code class="docutils literal notranslate"><span class="pre">Signature</span></code> object.</p>
<p>Both of the newer functions support keyword-only arguments and type
annotations:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">p1</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">p2</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="o">/</span><span class="p">,</span> <span class="n">kp3</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">kp4</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">10</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">kp5</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="n">k6</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">k7</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">12</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">k8</span><span class="p">:</span> <span class="nb">float</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
<span class="gp">... </span> <span class="k">pass</span>
<span class="gp">...</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">getfullargspec</span><span class="p">(</span><span class="n">foo</span><span class="p">)</span>
<span class="go">FullArgSpec(args=['p1', 'p2', 'kp3', 'kp4', 'kp5'],</span>
<span class="go"> varargs='args',</span>
<span class="go"> varkw='kwargs',</span>
<span class="go"> defaults=(10, None),</span>
<span class="go"> kwonlyargs=['k6', 'k7', 'k8'],</span>
<span class="go"> kwonlydefaults={'k7': 12},</span>
<span class="go"> annotations={'return': <class 'float'>,</span>
<span class="go"> 'p1': <class 'int'>,</span>
<span class="go"> 'p2': <class 'str'>,</span>
<span class="go"> 'kp3': <class 'str'>,</span>
<span class="go"> 'kp4': <class 'int'>,</span>
<span class="go"> 'kp5': <class 'float'>,</span>
<span class="go"> 'k6': <class 'str'>,</span>
<span class="go"> 'k7': <class 'int'>,</span>
<span class="go"> 'k8': <class 'float'>})</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">signature</span><span class="p">(</span><span class="n">foo</span><span class="p">)</span>
<span class="go"><Signature (p1: int, p2: str, /, kp3: str, kp4: int = 10,</span>
<span class="go"> kp5: float = None, *args, k6: str, k7: int = 12,</span>
<span class="go"> k8: float, **kwargs) -> float></span>
</pre></div>
</div>
<p>One notable difference between <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.signature">inspect.signature()</a> and the two other
functions is that the latter always include the ‘self’ argument
of method prototypes, while the former skips it if the method is bound
to an object. That is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">class</span> <span class="nc">foo</span><span class="p">:</span>
<span class="gp">... </span> <span class="k">def</span> <span class="nf">x</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">bar</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">pass</span>
<span class="gp">...</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">getargspec</span><span class="p">(</span><span class="n">foo</span><span class="o">.</span><span class="n">x</span><span class="p">)</span>
<span class="go">ArgSpec(args=['self', 'bar'], varargs=None, keywords=None, defaults=None)</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">getargspec</span><span class="p">(</span><span class="n">foo</span><span class="p">()</span><span class="o">.</span><span class="n">x</span><span class="p">)</span>
<span class="go">ArgSpec(args=['self', 'bar'], varargs=None, keywords=None, defaults=None)</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">signature</span><span class="p">(</span><span class="n">foo</span><span class="o">.</span><span class="n">x</span><span class="p">)</span>
<span class="go"><Signature (self, bar)></span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">signature</span><span class="p">(</span><span class="n">foo</span><span class="p">()</span><span class="o">.</span><span class="n">x</span><span class="p">)</span>
<span class="go"><Signature (bar)></span>
</pre></div>
</div>
<p>The <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.formatargspec">inspect.formatargspec()</a> function provides a pretty-formatted
argument spec from the tuple returned by <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.getfullargspec">inspect.getfullargspec()</a>
(or <a class="reference external" href="https://docs.python.org/3.10/library/inspect.html#inspect.getargspec">inspect.getargspec()</a>). It is replaced by stringification
of <code class="docutils literal notranslate"><span class="pre">Signature</span></code> objects:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">p1</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">p2</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="o">/</span><span class="p">,</span> <span class="n">kp3</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">kp4</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">10</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">kp5</span><span class="p">:</span> <span class="nb">float</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="n">k6</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">k7</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">12</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">k8</span><span class="p">:</span> <span class="nb">float</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span> <span class="o">-></span> <span class="nb">float</span><span class="p">:</span>
<span class="gp">... </span> <span class="k">pass</span>
<span class="gp">...</span>
<span class="gp">>>> </span><span class="n">inspect</span><span class="o">.</span><span class="n">formatargspec</span><span class="p">(</span><span class="o">*</span><span class="n">inspect</span><span class="o">.</span><span class="n">getfullargspec</span><span class="p">(</span><span class="n">foo</span><span class="p">))</span>
<span class="go">'(p1: int, p2: str, kp3: str, kp4: int=10, kp5: float=None, '</span>
<span class="go">'*args, k6: str, k7: int=12, k8: float, **kwargs) -> float'</span>
<span class="gp">>>> </span><span class="nb">str</span><span class="p">(</span><span class="n">inspect</span><span class="o">.</span><span class="n">signature</span><span class="p">(</span><span class="n">foo</span><span class="p">))</span>
<span class="go">'(p1: int, p2: str, /, kp3: str, kp4: int = 10, kp5: float = None, '</span>
<span class="go">'*args, k6: str, k7: int = 12, k8: float, **kwargs) -> float'</span>
</pre></div>
</div>
</section>
</section>
<section id="python-3-10">
<h2>Python 3.10<a class="headerlink" href="#python-3-10" title="Link to this heading">¶</a></h2>
<p>See also: <a class="reference external" href="https://docs.python.org/3/whatsnew/3.10.html">what’s new in Python 3.10</a></p>
<section id="configure-no-package-python-3-1-found">
<h3>configure: No package ‘python-3.1’ found<a class="headerlink" href="#configure-no-package-python-3-1-found" title="Link to this heading">¶</a></h3>
<p>automake prior to 1.16.3 wrongly recognized Python 3.10 as 3.1.
As a result, build with Python 3.10 fails:</p>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="go">checking for python version... 3.1</span>
<span class="go">checking for python platform... linux</span>
<span class="go">checking for python script directory... ${prefix}/lib/python3.10/site-packages</span>
<span class="go">checking for python extension module directory... ${exec_prefix}/lib/python3.10/site-packages</span>
<span class="go">checking for PYTHON... no</span>
<span class="go">configure: error: Package requirements (python-3.1) were not met:</span>
<span class="go">No package 'python-3.1' found</span>
<span class="go">Consider adjusting the PKG_CONFIG_PATH environment variable if you</span>
<span class="go">installed software in a non-standard prefix.</span>
<span class="go">Alternatively, you may set the environment variables PYTHON_CFLAGS</span>
<span class="go">and PYTHON_LIBS to avoid the need to call pkg-config.</span>
<span class="go">See the pkg-config man page for more details.</span>
<span class="go">Error: Process completed with exit code 1.</span>
</pre></div>
</div>
<p>To resolve this in ebuild, you need to autoreconf with the Gentoo
distribution of automake:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">inherit</span> <span class="n">autotools</span>
<span class="c1"># ...</span>
<span class="n">src_prepare</span><span class="p">()</span> <span class="p">{</span>
<span class="n">default</span>
<span class="n">eautoreconf</span>
<span class="p">}</span>
</pre></div>
</div>
<p>The upstream fix is to create new distfiles using automake-1.16.3+.</p>
</section>
<section id="distutils-sysconfig-deprecation">
<h3>distutils.sysconfig deprecation<a class="headerlink" href="#distutils-sysconfig-deprecation" title="Link to this heading">¶</a></h3>
<p>Upstream intends to remove distutils by Python 3.12. Python 3.10 starts
throwing deprecation warnings for various distutils modules.
The distutils.sysconfig is usually easy to port.</p>
<p>The following table summarizes replacements for common path getters.</p>
<blockquote>
<div><table class="docutils align-default">
<thead>
<tr class="row-odd"><th class="head"><p>distutils.sysconfig call</p></th>
<th class="head"><p>sysconfig replacement</p></th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">get_python_inc(False)</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">get_path("include")</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">get_python_inc(True)</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">get_path("platinclude")</span></code></p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">get_python_lib(False,</span> <span class="pre">False)</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">get_path("purelib")</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">get_python_lib(True,</span> <span class="pre">False)</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">get_path("platlib")</span></code></p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">get_python_lib(False,</span> <span class="pre">True)</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">get_path("stdlib")</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">get_python_lib(True,</span> <span class="pre">True)</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">get_path("platstdlib")</span></code></p></td>
</tr>
</tbody>
</table>
</div></blockquote>
<p>For both functions, omitted parameters default to <code class="docutils literal notranslate"><span class="pre">False</span></code>. There is
no trivial replacement for the variants with <code class="docutils literal notranslate"><span class="pre">prefix</span></code> argument.</p>
</section>
</section>
<section id="python-3-9">
<h2>Python 3.9<a class="headerlink" href="#python-3-9" title="Link to this heading">¶</a></h2>
<p>See also: <a class="reference external" href="https://docs.python.org/3/whatsnew/3.9.html">what’s new in Python 3.9</a></p>
<section id="base64-encodestring-base64-decodestring-removal">
<h3>base64.encodestring / base64.decodestring removal<a class="headerlink" href="#base64-encodestring-base64-decodestring-removal" title="Link to this heading">¶</a></h3>
<p>Python 3.9 removes the deprecated <code class="docutils literal notranslate"><span class="pre">base64.encodestring()</span></code>
and <code class="docutils literal notranslate"><span class="pre">base64.decodestring()</span></code> functions. While they were deprecated
since Python 3.1, many packages still use them today.</p>
<p>The drop-in Python 3.1+ replacements are <code class="docutils literal notranslate"><span class="pre">base64.encodebytes()</span></code>
and <code class="docutils literal notranslate"><span class="pre">base64.decodebytes()</span></code>. Note that contrary to the names, the old
functions were simply aliases to the byte variants in Python 3
and <em>required</em> the arguments to be <code class="docutils literal notranslate"><span class="pre">bytes</span></code> anyway.</p>
<p>If compatibility with Python 2 is still desired, then the byte variants
ought to be called on 3.1+ and string variants before that. The old
variants accept both byte and unicode strings on Python 2.</p>
<p>Example compatibility import:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">if</span> <span class="n">sys</span><span class="o">.</span><span class="n">version_info</span> <span class="o">>=</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">):</span>
<span class="kn">from</span> <span class="nn">base64</span> <span class="kn">import</span> <span class="n">encodebytes</span> <span class="k">as</span> <span class="n">b64_encodebytes</span>
<span class="k">else</span><span class="p">:</span>
<span class="kn">from</span> <span class="nn">base64</span> <span class="kn">import</span> <span class="n">encodestring</span> <span class="k">as</span> <span class="n">b64_encodebytes</span>
</pre></div>
</div>
<p>Note that the <code class="docutils literal notranslate"><span class="pre">base64</span></code> module also provides <code class="docutils literal notranslate"><span class="pre">b64encode()</span></code>
and <code class="docutils literal notranslate"><span class="pre">b64decode()</span></code> functions that were not renamed. <code class="docutils literal notranslate"><span class="pre">b64decode()</span></code>
can be used as a drop-in replacement for <code class="docutils literal notranslate"><span class="pre">decodebytes()</span></code>. However,
<code class="docutils literal notranslate"><span class="pre">b64encode()</span></code> does not insert newlines to split the output
like <code class="docutils literal notranslate"><span class="pre">encodebytes()</span></code> does, and instead returns a single line
of base64-encoded data for any length of output.</p>
</section>
</section>
<section id="python-3-8">
<h2>Python 3.8<a class="headerlink" href="#python-3-8" title="Link to this heading">¶</a></h2>
<p>See also: <a class="reference external" href="https://docs.python.org/3/whatsnew/3.8.html">what’s new in Python 3.8</a></p>
<section id="python-config-and-pkg-config-no-longer-list-python-library-by-default">
<h3>python-config and pkg-config no longer list Python library by default<a class="headerlink" href="#python-config-and-pkg-config-no-longer-list-python-library-by-default" title="Link to this heading">¶</a></h3>
<p>Until Python 3.7, the <code class="docutils literal notranslate"><span class="pre">python-X.Y</span></code> pkg-config file and python-config
tool listed the Python library. Starting with 3.8, this is no longer
the case. If you are building Python extensions, this is fine (they
are not supposed to link directly to libpython).</p>
<p>If you are building programs that need to embed the Python interpreter,
new <code class="docutils literal notranslate"><span class="pre">python-X.Y-embed</span></code> pkg-config file and <code class="docutils literal notranslate"><span class="pre">--embed</span></code> parameter
are provided for the purpose.</p>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>pkg-config<span class="w"> </span>--libs<span class="w"> </span>python-3.7
<span class="go">-lpython3.7m</span>
<span class="gp">$ </span>pkg-config<span class="w"> </span>--libs<span class="w"> </span>python-3.8
<span class="gp">$ </span>pkg-config<span class="w"> </span>--libs<span class="w"> </span>python-3.8-embed
<span class="go">-lpython3.8</span>
</pre></div>
</div>
<p>To achieve backwards compatibility, you should query
<code class="docutils literal notranslate"><span class="pre">python-X.Y-embed</span></code> first and fall back to <code class="docutils literal notranslate"><span class="pre">python-X.Y</span></code>.</p>
</section>
</section>
<section id="replacing-the-toml-package">
<h2>Replacing the toml package<a class="headerlink" href="#replacing-the-toml-package" title="Link to this heading">¶</a></h2>
<p>The old <a class="reference external" href="https://pypi.org/project/toml/">toml</a> package is no longer maintained. It was last released
in November 2020 and it was never updated to implement TOML 1.0.
The recommended alternatives are:</p>
<ul class="simple">
<li><p>the built-in <a class="reference external" href="https://docs.python.org/3.11/library/tomllib.html">tomllib</a> module (since Python 3.11) with fallback to
<a class="reference external" href="https://pypi.org/project/tomli/">tomli</a> package for reading TOML files</p></li>
<li><p>the <a class="reference external" href="https://pypi.org/project/tomli-w/">tomli-w</a> package for writing TOML files</p></li>
<li><p>the <a class="reference external" href="https://pypi.org/project/tomlkit/">tomlkit</a> package for editing already existing TOML files
while preserving style</p></li>
</ul>
<section id="porting-to-tomllib-tomli-without-toml-fallback">
<h3>Porting to tomllib/tomli without toml fallback<a class="headerlink" href="#porting-to-tomllib-tomli-without-toml-fallback" title="Link to this heading">¶</a></h3>
<p>Using a combination of <a class="reference external" href="https://docs.python.org/3.11/library/tomllib.html">tomllib</a> and <a class="reference external" href="https://pypi.org/project/tomli/">tomli</a> is the recommended approach
for packages that only read TOML files, or both read and write them
but do not need to preserve style. The tomllib module is available
since Python 3.11, while tomli versions providing a compatible API
are compatible with Python 3.6 and newer.</p>
<p>The key differences between <a class="reference external" href="https://pypi.org/project/toml/">toml</a> and tomllib/tomli are:</p>
<ul class="simple">
<li><p>the <code class="docutils literal notranslate"><span class="pre">load()</span></code> function accepts only a file object open for reading
in binary mode whereas toml expects a path or a file object open
for reading in text mode</p></li>
<li><p>the exception raised for invalid input is named <code class="docutils literal notranslate"><span class="pre">TOMLDecodeError</span></code>
where it is named <code class="docutils literal notranslate"><span class="pre">TomlDecodeError</span></code> in toml</p></li>
</ul>
<p>For example, the following code:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">toml</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">d1</span> <span class="o">=</span> <span class="n">toml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s2">"in1.toml"</span><span class="p">)</span>
<span class="k">except</span> <span class="n">toml</span><span class="o">.</span><span class="n">TomlDecodeError</span><span class="p">:</span>
<span class="n">d1</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"in2.toml"</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">d2</span> <span class="o">=</span> <span class="n">toml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">d3</span> <span class="o">=</span> <span class="n">toml</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="s1">'test = "foo"</span><span class="se">\n</span><span class="s1">'</span><span class="p">)</span>
</pre></div>
</div>
<p>would normally be written as:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">if</span> <span class="n">sys</span><span class="o">.</span><span class="n">version_info</span> <span class="o">>=</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">11</span><span class="p">):</span>
<span class="kn">import</span> <span class="nn">tomllib</span>
<span class="k">else</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">tomli</span> <span class="k">as</span> <span class="nn">tomllib</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># tomllib does not accept paths</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"in1.toml"</span><span class="p">,</span> <span class="s2">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">d1</span> <span class="o">=</span> <span class="n">tomllib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="c1"># the exception uses uppercase "TOML"</span>
<span class="k">except</span> <span class="n">tomllib</span><span class="o">.</span><span class="n">TOMLDecodeError</span><span class="p">:</span>
<span class="n">d1</span> <span class="o">=</span> <span class="kc">None</span>
<span class="c1"># the file must be open in binary mode</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"in2.toml"</span><span class="p">,</span> <span class="s2">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">d2</span> <span class="o">=</span> <span class="n">tomllib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">d3</span> <span class="o">=</span> <span class="n">tomllib</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="s1">'test = "foo"</span><span class="se">\n</span><span class="s1">'</span><span class="p">)</span>
</pre></div>
</div>
<p>The following dependency string:</p>
<div class="highlight-toml notranslate"><div class="highlight"><pre><span></span><span class="n">dependencies</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span>
<span class="w"> </span><span class="s2">"toml"</span><span class="p">,</span>
<span class="p">]</span>
</pre></div>
</div>
<p>would be replaced by:</p>
<div class="highlight-toml notranslate"><div class="highlight"><pre><span></span><span class="n">dependencies</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span>
<span class="w"> </span><span class="s2">"tomli >= 1.2.3; python_version < '3.11'"</span><span class="p">,</span>
<span class="p">]</span>
</pre></div>
</div>
</section>
<section id="porting-to-tomllib-tomli-with-toml-fallback">
<h3>Porting to tomllib/tomli with toml fallback<a class="headerlink" href="#porting-to-tomllib-tomli-with-toml-fallback" title="Link to this heading">¶</a></h3>
<p>If upstream insists on preserving compatibility with EOL versions
of Python, it is possible to use a combination of <a class="reference external" href="https://docs.python.org/3.11/library/tomllib.html">tomllib</a>, <a class="reference external" href="https://pypi.org/project/tomli/">tomli</a>
and <a class="reference external" href="https://pypi.org/project/toml/">toml</a>. Unfortunately, the incompatibilites in API need to be taken
into consideration.</p>
<p>For example, a backwards compatible code for loading a TOML file could
look like the following:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">if</span> <span class="n">sys</span><span class="o">.</span><span class="n">version_info</span> <span class="o">>=</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">11</span><span class="p">):</span>
<span class="kn">import</span> <span class="nn">tomllib</span>
<span class="k">else</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">tomli</span> <span class="k">as</span> <span class="nn">tomllib</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"in1.toml"</span><span class="p">,</span> <span class="s2">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">d1</span> <span class="o">=</span> <span class="n">tomllib</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="k">except</span> <span class="n">tomllib</span><span class="o">.</span><span class="n">TOMLDecodeError</span><span class="p">:</span>
<span class="n">d1</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">toml</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"in1.toml"</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">d1</span> <span class="o">=</span> <span class="n">toml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="k">except</span> <span class="n">toml</span><span class="o">.</span><span class="n">TomlDecodeError</span><span class="p">:</span>
<span class="n">d1</span> <span class="o">=</span> <span class="kc">None</span>
</pre></div>
</div>
<p>In this case, the dependency string becomes more complex:</p>
<div class="highlight-toml notranslate"><div class="highlight"><pre><span></span><span class="n">dependencies</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span>
<span class="w"> </span><span class="s2">"tomli >= 1.2.3; python_version >= '3.6' and python_version < '3.11'"</span><span class="p">,</span>
<span class="w"> </span><span class="s2">"toml; python_version < '3.6'"</span><span class="p">,</span>
<span class="p">]</span>
</pre></div>
</div>
</section>
<section id="porting-to-tomli-w">
<h3>Porting to tomli-w<a class="headerlink" href="#porting-to-tomli-w" title="Link to this heading">¶</a></h3>
<p><a class="reference external" href="https://pypi.org/project/tomli-w/">tomli-w</a> provides a minimal module for dumping TOML files.</p>
<p>The key differences between <a class="reference external" href="https://pypi.org/project/toml/">toml</a> and tomli-w are:</p>
<ul class="simple">
<li><p>the <code class="docutils literal notranslate"><span class="pre">dump()</span></code> function takes a file object open for writing in binary
mode whereas toml expected a file object open for writing in text mode</p></li>
<li><p>providing a custom encoder instance is not supported</p></li>
</ul>
<p>For example, the following code:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">toml</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"out.toml"</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">toml</span><span class="o">.</span><span class="n">dump</span><span class="p">({</span><span class="s2">"test"</span><span class="p">:</span> <span class="s2">"data"</span><span class="p">},</span> <span class="n">f</span><span class="p">)</span>
</pre></div>
</div>
<p>would be replaced by:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">tomli_w</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"out.toml"</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">tomli_w</span><span class="o">.</span><span class="n">dump</span><span class="p">({</span><span class="s2">"test"</span><span class="p">:</span> <span class="s2">"data"</span><span class="p">},</span> <span class="n">f</span><span class="p">)</span>
</pre></div>
</div>
<p>Note that when both reading and writing TOML files is necessary, two
modules need to be imported and used separately rather than one.</p>
</section>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper">
<h1 class="logo"><a href="index.html">Gentoo Python Guide</a></h1>
<search id="searchbox" style="display: none" role="search">
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false" placeholder="Search"/>
<input type="submit" value="Go" />
</form>
</div>
</search>
<script>document.getElementById('searchbox').style.display = "block"</script><h3>Navigation</h3>
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="preface.html">Preface</a></li>
<li class="toctree-l1"><a class="reference internal" href="interpreter.html">Python interpreters</a></li>
<li class="toctree-l1"><a class="reference internal" href="eclass.html">Choosing between Python eclasses</a></li>
<li class="toctree-l1"><a class="reference internal" href="basic.html">Common basics</a></li>
<li class="toctree-l1"><a class="reference internal" href="any.html">python-any-r1 — build-time dependency</a></li>
<li class="toctree-l1"><a class="reference internal" href="single.html">python-single-r1 — single-impl packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="multi.html">python-r1 — multi-impl packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils.html">distutils-r1 — standard Python build systems</a></li>
<li class="toctree-l1"><a class="reference internal" href="test.html">Tests in Python packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils-legacy.html">distutils-r1 legacy concepts</a></li>
<li class="toctree-l1"><a class="reference internal" href="pypi.html">pypi — helper eclass for PyPI archives</a></li>
<li class="toctree-l1"><a class="reference internal" href="helper.html">Common helper functions</a></li>
<li class="toctree-l1"><a class="reference internal" href="depend.html">Advanced dependencies</a></li>
<li class="toctree-l1"><a class="reference internal" href="pytest.html">pytest recipes</a></li>
<li class="toctree-l1"><a class="reference internal" href="concept.html">Advanced concepts</a></li>
<li class="toctree-l1"><a class="reference internal" href="expert-multi.html">Expert python-r1 usage</a></li>
<li class="toctree-l1"><a class="reference internal" href="buildsys.html">Integration with build systems written in Python</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Porting tips</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#retroactive-changes">Retroactive changes</a></li>
<li class="toctree-l2"><a class="reference internal" href="#experimental-python-implementations">Experimental Python implementations</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-3-13">Python 3.13</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-3-12">Python 3.12</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-3-11">Python 3.11</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-3-10">Python 3.10</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-3-9">Python 3.9</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-3-8">Python 3.8</a></li>
<li class="toctree-l2"><a class="reference internal" href="#replacing-the-toml-package">Replacing the toml package</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="migration.html">Migration guides</a></li>
<li class="toctree-l1"><a class="reference internal" href="qawarn.html">QA checks and warnings</a></li>
<li class="toctree-l1"><a class="reference internal" href="package-maintenance.html">Python package maintenance</a></li>
<li class="toctree-l1"><a class="reference internal" href="interpreter-maintenance.html">Maintenance of Python implementations</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="buildsys.html" title="previous chapter">Integration with build systems written in Python</a></li>
<li>Next: <a href="migration.html" title="next chapter">Migration guides</a></li>
</ul></li>
</ul>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2020, Michał Górny, license: CC BY 4.0.
|
Powered by <a href="https://www.sphinx-doc.org/">Sphinx 8.1.3</a>
& <a href="https://alabaster.readthedocs.io">Alabaster 1.0.0</a>
|
<a href="_sources/porting.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>
|
GitWeb