Metabug: Improving C-level coverage

Comienza con la guía de coverage enlazada y ejecuta coverage localmente usando el árbol actual de CPython. Elige un archivo fuente sin marcar o una brecha indicada, inspecciona el punto de entrada correspondiente en C y las pruebas existentes, y después registra cualquier trabajo de coverage específico como un subelemento o un issue enlazado. Se considera terminado cuando la brecha seleccionada está probada y la lista de comprobación está actualizada.

[edit by @encukou, May 2024] The coverage report and checklist are outdated. Run coverage locally before contributing.

---

This bug is going to be used to track work in a other bugs to improve the C-level coverage of the CPython test suite.

There is a set of baseline coverage results on main [edit: outdated, see above] that can be used to find coverage gaps.

The plan, discussed on discuss.python.org is as follows:

• Read through the coverage report and record any notable gaps in the checklist below. The goal is not 100% coverage, and each area of improvement will probably require some judgement calls. For example, covering all cases where memory exhaustion can occur is probably not worth the effort. On the other hand, detailed coverage in the eval loop may be worth the effort.
• When someone has "read through" a particular source file and added created subitems for any interesting gaps, they should check it off on the list below and add links to any issues created.

Related work:

There is related work to publish coverage results from CPython on a regular basis, but this issue is concerned with using those results to actually reduce our gaps in coverage.

List of source files:

• Include/internal/pycore_asdl.h
• Include/internal/pycore_bitutils.h
• Include/internal/pycore_call.h
• Include/internal/pycore_code.h
• Include/internal/pycore_frame.h
• Include/internal/pycore_moduleobject.h
• Include/internal/pycore_object.h
• Include/internal/pycore_pymath.h
• Include/internal/pycore_pymem.h
• Include/internal/pycore_pystate.h
• Include/object.h
• Include/pydtrace.h
• Objects/abstract.c
  • Buffer related functions: PyBuffer_FromContiguous, PyObject_CopyData, PyBuffer_FillContiguousStrides
  • PyNumber_Check doesn't test complex
  • PySequence_Repeat and PySequence_InPlaceRepeat have no coverage
  • PySequence_SetItem with a negative index is untested
  • PySequence_SetSlice and PySequence_DelSlice are untested
  • PyMapping_HasKey and PyMapping_HasKeyString are untested
• Objects/accu.c
• Objects/boolobject.c
  • #94859
• Objects/bytearrayobject.c
  • #95802
• Objects/bytes_methods.c
• Objects/bytesobject.c
  • In PyBytes_FromFormatV, the special handling of %p isn't tested. It's only the case where the underlying libc is broken that isn't tested.
  • #95895
• Objects/call.c
  • PyEval_CallObjectWithKeywords has no coverage
  • _PyObject_CallMethodId_SizeT has no coverage
• Objects/capsule.c
• Objects/cellobject.c
• Objects/classobject.c
• Objects/codeobject.c
  • #94814
  • #96609
  • #94816
• Objects/complexobject.c
• Objects/descrobject.c
• Objects/dictobject.c
  • In dictresize convert split table into new combined table" is uncovered.
  • _PyDict_GetItemHint has no coverage
• Objects/enumobject.c
• Objects/exceptions.c
• Objects/fileobject.c
  • PyFile_FromFd has no coverage
  • PyFile_GetLine over bytes input has no coverage
• Objects/floatobject.c
  • #94860
• Objects/frameobject.c
  • frame_setlineno has poor coverage in its helper functions get_arg and mark_stacks.
  • _PyFrame_GetState has a switch statement where only the default case is covered.
  • In _PyFrame_FastToLocalsWithError there is no test that exercises the COPY_FREE_VARS case.
• Objects/funcobject.c
  • A bunch of API is untested: PyFunction_GetCode, PyFunction_GetGlobals, PyFunction_GetModule, PyFunction_GetDefaults, PyFunction_SetDefaults, PyFunction_GetKwDefaults, PyFunction_SetKwDefaults, PyFunction_GetClosure, PyFunction_SetClosure, PyFunction_GetAnnotations, PyFunction_SetAnnotations \
  • #98449
  • #98317
• Objects/genericaliasobject.c
• Objects/genobject.c
  • gen_new_with_qualname and API PyGen_NewWithQualName and PyGen_New have no coverage.
  • PyCoro_New has no coverage
  • PyAsyncGen_New has no coverage
  • async_gen_athrow_send has poor coverage
• Objects/interpreteridobject.c
• Objects/iterobject.c
  • #95923
• Objects/listobject.c
• Objects/longobject.c
  • _PyLong_Sing_t_Converter has no coverage
  • long_format_binary doesn't test outputting to UCS2 or UCS4
  • int_bit_length_impl and int_bit_count_impl doesn't cover the case where expression overflows
• Objects/memoryobject.c
  • init_slice is not well-covered
• Objects/methodobject.c
• Objects/moduleobject.c
  • PyModule_GetFilename has no coverage
• Objects/namespaceobject.c
• Objects/object.c
  • PyObject_Print has no coverage
  • PyObject_Bytes does not test the case where there is a __bytes__
  • #96627
  • PyObject_SetAttrString doesn't test when object has a tp_setattr
  • PyObject_GetAttrString doesn't test when object has a tp_getattr
  • _PyObject_LookupAttr doesn't test when object has a tp_getattr
• Objects/obmalloc.c
• Objects/odictobject.c
• Objects/picklebufobject.c
  • PyPickleBuffer_FromObject, PyPickleBuffer_Release has no coverage
• Objects/rangeobject.c
• Objects/setobject.c
• Objects/sliceobject.c
  • PySlice_GetIndices/PySlice_GetIndicesEx has no coverage
• Objects/stringlib/codecs.h
• Objects/stringlib/count.h
• Objects/stringlib/ctype.h
• Objects/stringlib/eq.h
• Objects/stringlib/fastsearch.h
  • #96760
• Objects/stringlib/find.h
• Objects/stringlib/find_max_char.h
• Objects/stringlib/join.h
• Objects/stringlib/localeutil.h
• Objects/stringlib/partition.h
• Objects/stringlib/replace.h
• Objects/stringlib/split.h
• Objects/stringlib/transmogrify.h
• Objects/stringlib/undef.h
• Objects/stringlib/unicode_format.h
• Objects/structseq.c
• Objects/tupleobject.c
• Objects/typeobject.c
  • wrap_sq_setitem has no coverage
• Objects/unicodectype.c
• Objects/unicodeobject.c
  • xmlcharrefreplace doesn't test for codepoints < 100 (This seems almost impossible to occur).
  • resize_inplace has no coverage
  • unicode_kind_name when !PyUnicode_IS_COMPACT isn't covered -- low priority used by consistency check only
  • unicode_write_cstr doesn't test writing into UCS2 or UCS4
  • #96677
  • PyUnicode_AsDecodedObject, PyUnicode_AsDecodedUnicode, PyUnicode_AsEncodedObject, PyUnicode_AsEncodedUnicode has no coverage
  • _Py_DecodeUTF8Ex and _Py_EncodeUTF8Ex has no coverage for error == surrogateescape
  • PyUnicode_BuildEncodingMap doesn't handle the need_dict case
  • ucs1lib_find_slice and ucs1lib_rfind_slice aren't covered.
  • PyUnicode_Count has no coverage
  • #98228
  • PyUnicode_CompareWithASCIIString has no coverage for comparing with UCS2 or UCS4
  • _PyUnicode_EqualToASCIIId has no coverage
• Objects/unicodetype_db.h
• Objects/unionobject.c
• Objects/weakrefobject.c
• Parser/action_helpers.c
  • _PyPegen_set_expr_context doesn't cover "starred kind"
  • _PyPegen_get_expr_name switch statement coverage is non-exhaustive
• Parser/myreadline.c (N/A Windows-only)
• Parser/parser.c
• Parser/peg_api.c
• Parser/pegen.c
• Parser/pegen.h
• Parser/pegen_errors.c
  • #94926
• Parser/string_parser.c
  • #95925
• Parser/tokenizer.c
  • #94823
  • tokenizer.c seems to have no coverage for a few functions related to interactive usage, e.g. tok_underflow_interactive and tok_concatenate_interactive_newline.
• Python/Python-ast.c Generated code
• Python/Python-tokenize.c
• Python/_warnings.c
  • show_warning doesn't cover the case where there is a sourceline.
  • PyErr_WarnExplicit has no coverage
• Python/asdl.c
• Python/ast.c
  • ensure_literal_* functions aren't covered
  • validate_pattern_match_value doesn't cover all elements of switch
• Python/ast_opt.c
  • check_complexity doesn't cover the frozenset case
  • ast_foldbody isn't covered
• Python/ast_unparse.c
• Python/bltinmodule.c
• Python/bootstrap_hash.c
• Python/ceval.c
  • PyEval_AquireLock and PyEval_ReleaseLock are uncovered
  • #95932
  • STORE_ATTR_WITH_HINT doesn't cover the case where the dictionary doesn't have Unicode keys
  • CALL_FUNCTION_EX doesn't cover the case where kwargs is not an exact dict
  • PyEval_EvalCodeEx doesn't cover the case where kwargs are passed in
  • PyEval_GetFrame has no coverage
  • #98300
• Python/ceval_gil.h
• Python/codecs.c
• Python/compile.c
  • write_instr is not handling the case where ilen > 2. It might be that those are never seen in practice...? If so, feel free to close this bug.
  • check_ann_subscr doesn't have any coverage for slice or tuple kinds.
  • optimize_basic_block has some opcodes that aren't covered in the JUMP_IF_FALSE_OR_POP and the JUMP_IF_TRUE_OR_POP cases.
• Python/condvar.h
• Python/context.c
  • PyContext_Copy, PyContext_Enter, PyContext_Exit have no coverage
• Python/deepfreeze/deepfreeze.c
• Python/dtoa.c
• Python/dup2.c
• Python/dynamic_annotations.c
• Python/errors.c
• Python/fileutils.c
  • is_valid_wide_char doesn't test error branches
  • encode_ascii/decode_ascii has no coverage (probably very low priority -- comment says only for platforms with a broken mbstowcs (FreeBSD, OpenIndiana)
  • _Py_stat has no coverage
• Python/formatter_unicode.c
• Python/frame.c
• Python/frozenmain.c
• Python/future.c
• Python/getargs.c
• Python/getopt.c
• Python/hamt.c
• Python/hashtable.c
• Python/import.c
• Python/importdl.c
• Python/initconfig.c
• Python/marshal.c
• Python/modsupport.c
• Python/mysnprintf.c
• Python/mystrtoul.c
• Python/pathconfig.c
• Python/preconfig.c
• Python/pyarena.c
• Python/pyfpe.c
• Python/pyhash.c
• Python/pylifecycle.c
• Python/pystate.c
• Python/pystrcmp.c
• Python/pystrhex.c
• Python/pystrtod.c
• Python/pythonrun.c
• Python/pytime.c
• Python/specialize.c
• Python/structmember.c
• Python/suggestions.c
• Python/symtable.c
• Python/sysmodule.c
• Python/thread.c
• Python/traceback.c
  • tracebacks with angle-bracketed filenames https://github.com/python/cpython/issues/95259
  • tb_printinternal with depth > limit
  • _PyTraceBack_Print_Indented with overflowing tracebacklimit
  • No coverage for _Py_DumpDecimal, _Py_DumpHexadecimal, _Py_DumpASCII, dump_frame, dump_traceback, _Py_DumpTraceback, write_thread_id, _Py_DumpTracebackThreads -- possibly they have tests which are disabled under some circumstances.

• PR: gh-98749

• PR: gh-96767

• PR: gh-99091

• PR: gh-99123

• PR: gh-98300

• PR: gh-98809

• PR: gh-99126

• PR: gh-99133

• PR: gh-99196

• PR: gh-99319

• PR: gh-99429

• PR: gh-99472

• PR: gh-97672

• PR: gh-100483

• PR: gh-100484

• PR: gh-100619

• PR: gh-102469

• PR: gh-111445

• PR: gh-117421

• PR: gh-119222

• PR: gh-119227

• PR: gh-119263

• PR: gh-119264

• PR: gh-138866

• PR: gh-156966