Python

The main articles are:

• Logical programming
• Evidence-based programming
• Programming methodology
• Programming technology
• Programmer

Python is a high-level general-purpose programming language focused on improving developer performance and code readability. Python kernel syntax is minimalistic. At the same time, the standard library includes a large set of useful functions.

Python supports structural, generalized, object-oriented, functional, and aspect-oriented programming. The main architectural features are dynamic typing, automatic memory management, complete introspection, exception handling mechanism, support for multithreaded computing, high-level data structures. It supports the division of programs into modules, which, in turn, can be combined into packages.

The reference implementation of Python is the CPython interpreter, which supports most actively used platforms. It is distributed under the Python Software Foundation License, which allows it to be used without restrictions in any applications, including proprietary ones. There is an interpreter implementation for compilation-capable JVM, CLR, LLVM, other independent implementations. The PyPy project uses JIT compilation, which significantly increases the execution speed of Python programs.

Python is an actively developing programming language, new versions with the addition/change of language properties are released approximately every two and a half years. The language was not officially standardized, the role of the standard is de facto performed by CPython, developed under the control of the author of the language.

2024: Convenient programming language for backend development

Choosing a language programming for backend development plays a key role in the success of the project. There IT are many tools for writing codes and each has its own features and applications. Among the most popular languages ​ ​ are,, Java JavaScript Python,,. Ruby C# If the project is new and there is an opportunity to choose, then it is worth paying attention to Python, which is gaining weight through development, machine learning says Kirill Shershen, an experienced IT specialist, leading American backend developer, startup winner of the main award of the National Prize "Technologies and Innovations 2023," leading expert of the Association of Technical Innovations - IEEE Institute of Engineers. electrical engineering and electronics Read more here. [1]

2023

About 1,000 libraries with a vulnerability that can infect legitimate software

On December 1, 2023, MTS RED, a member of MTS PJSC, published the results of a study of more than 4.7 million unique public program code repositories. The MTS RED ART (Advanced Research Team) research team has identified about 1000 vulnerable libraries that attackers can use to infect legitimate software with malware. As representatives of MTS RED clarified TAdviser, the libraries studied as part of the study are used in the development of software in the Python programming language.

In development, the use of external components from third-party developers - libraries is widespread. If they are connected from an external repository, then there is a risk of an attacker seizing this code store - an attack such as repository hijacking or repojacking.

Vulnerability when using external components, it occurs if their developer has transferred, deleted or sold their account - for example, due to the termination of project support or the transfer of the repository with the code to the customer. In some cases, attackers are able to register a "abandoned" account name and place malicious code in it. In this case, programs using the component on the old link will already access the new content controlled by the attacker. Thus, code with various malicious functions can be built into a legitimate and popular program unnoticed by the developer - from theft data and tracking users to complete control over user devices.

MTS RED ART experts analyzed over 6.3 million repositories from such popular sources as Google Cloud Console, PyPi, NPM. The analysis made it possible to allocate 4.7 million unique repositories, according to links to 7820 of them there is no content. In 986 cases, the account under which repositories were originally created is inactive. Such repositories are vulnerable to capture by attackers who can re-register them for themselves, including using automation tools.

Accounts vulnerable to repojacking class cyber attacks could be used in a large number of programs, including Russian developers, who are unaware of the potential danger. To prevent third-party components from becoming the entry point for a hacker attack on developers, MTS RED ART has published a methodology and links to a number of tools that allow you to check the source code for the correctness of links to the external repositories used.

We invite the developer community to join the initiative and once again check for direct links to third-party repositories in their dependencies that may be vulnerable to repojacking attacks. Our common goal is to create a register of trusted dependencies in which all third-party components and their artifacts are verified by the community, "said Denis Makrushin, Technical Director of MTS RED.

Introduced software that speeds up the work of programs written in Python by thousands of times

In late August 2023, researchers at the University of Massachusetts Amherst announced the creation of the Scalene profiler, which allows you to repeatedly speed up the work of programs written in Python. Read more here.

Fix a vulnerability in the TLS implementation

Corrective updates to the Python programming language 3.11.5, 3.10.13, 3.9.18 and 3.8.18 have been published, which fixed the vulnerability (CVE-2023-40217) in the ssl class. SSLSocket, which allows you to bypass the TLS connection negotiation stage and related processes, such as certificate validation. A successful attack can process unencrypted data as if it were transmitted using a valid TLS connection. This became known on August 25, 2023.

The problem is caused by the fact that after creating the socket, there is a small window during which the data received and placed in the socket buffer will be processed as read from the client if the connection is closed before the TLS connection negotiation process begins. To carry out an attack, it is enough to establish a connection, immediately send data and close the socket, without waiting for a response to negotiate a TLS connection. The size of data that can be sent during an attack is limited by the size of the network buffer.

The vulnerability affects server applications (for example, HTTPSservers-) that use the standard Python ssl library to organize a secure communication channel that uses authentication client certificates (for example, mTLS). The vulnerability can only be used to send data bypassing authentications certificates - since the connection is immediately closed, the response to the request will not be sent to the client. At the same time, the vulnerability may well be used for attacks on, API through which you can make changes or delete data.

The vulnerability could also be exploited to attack clients connected to an attacker-controlled server if those clients immediately proceed to read data from the socket, without first sending a request (regular client applications such as pip, which use HTTPS to send requests, the vulnerability does not affect).

In addition, another vulnerability (CVE-2023-41105) has been fixed in the Python 3.11 branch, which allows bypassing valid file path checks using the os.path.normpath () function. The vulnerability is caused by the fact that if characters with zero code ('\0 ') are present in the path, the os.path.normpath () function truncates the path after the first zero character. At the same time, in subsequent functions of working with files, a full path can be used, not a cropped path. The problem manifests itself only in the 3.11.x branch, i.e. code based on os.path.normpath (), which correctly blocks invalid paths when executed in Python 3.10.x, can be bypassed when this code is executed in Python^[1]TLS^[2].

Integration in Excel

The company, Microsoft which since 2020 employs Guido van Rossum, the creator programming of the Python language, announced integration Python in a tabular form. processor Excel Python can be used in Excel to write formulas, work with, data analyze information , and form diagrams. Python code is added to cells, processed by analogy with macros and functions, but executed not locally, cloud but in Microsoft Cloud. This became known on August 22, 2023. More. here

Handler support in ispmanager beta relysis 6.74

In the beta release 6.74 of the ispmanager control panel 6 dated May 16, support for the Python handler appeared. Ispmanager (ISPmanager) announced this on May 18, 2023. This feature is available for lite, host, and pro versions. Read more here.

Creating a tool to automatically correct errors in Python scripts

In mid-March 2023, developer BioBootloader released a specialized tool for automatic error correction in Python scripts. The solution was called Wolverine. Read more here.

2022

Python 3.11

On October 24, 2022, it became known that after a year of development, a significant update to the Python programming language was published (with version 3.11). The updated branch will be supported for a year and a half, after which fixes will be formed for it for another three and a half years with the elimination of vulnerabilities.

Python 3.11

As reported, alpha testing of the Python 3.12 branch began at the same time (in accordance with the development schedule, work on the updated branch begins five months before the release of the previous branch and by the time of the next release reaches the alpha testing stage). The Python 3.12 branch will be at the alpha release stage for seven months, during which features will be added and errors fixed. After that, beta versions will be tested for three months, during which adding features will be prohibited and all attention will be paid to fixing bugs. The last two months before the release, the branch will be at the stage of a candidate for releases, at which the final stabilization will be carried out.

Among the changes added to Python 3.11 are:

• Performance optimization has been carried out. The updated branch includes changes related to acceleration and inline-deployment of the function call, the use of fast interpreters of typical operations (x + x, x * x, x-x, a [i], a [i] = z, f (arg) C (arg), o.method (), o.attr = z, * seq), as well as optimizations prepared by Cinder and HotPy projects. Depending on the type of load, there is an increase in the speed of code execution by 10-60%. On average, performance increased by 25% when passing the pyperformance test kit.
• The byte code caching mechanism has been redesigned, which reduced the start-up time of the interpreter by 10-15%. Objects with code and byte code are now statically placed by the interpreter, which made it possible to exclude the stages of demarchaling the byte code extracted from the cache and converting objects with code for placement in dynamic memory.
• When displaying call tracing in diagnostic messages information , an expression is output, due to which an error occurred (previously only a line without detail was highlighted, which part of the line caused the error). Advanced trace information can also be obtained through API and used to map individual byte code statements to a specific position in the source code using the codeobject.co_positions () method or the C function of the PyCode_Addr2Location () API. The change optimizes debugging of problems associated with nested dictionary objects, multiple function calls, and complex arithmetic expressions.
• Added support for exception groups that allow the program to generate and process several different exceptions at once. Additional ExceptionGroup and BaseExceptionGroup exception types are proposed to group multiple exceptions and call them together, and the expression "except *" is added to select individual exceptions from the group.
• The add_note () method has been added to the BaseException class, which allows you to attach a text note to an exception, for example, add contextual information that is not available during the generation of an exception.
• A special Self type has been added that represents the current private class. Self can be used to annotate methods that return an instance of their class in a simpler way than when using TypeVar.

• A special type LiteralString has been added, which can only include string literals that are compatible with the type LiteralString (that is, bare strings and strings with the type LiteralString, but not arbitrary and not combined strings with the type str). The type LiteralString can be used to restrict the transmission of string arguments to functions, arbitrary substitution of parts of strings in which can lead to vulnerabilities, for example, when generating strings for SQL queries or shell commands.
• Added TypeVarTuple, which allows you to use variable generics, unlike TypeVar, covering not one type, but an arbitrary number of types.
• The standard library includes a tomllib module with functions for parsing the TOML format.
• It is possible to mark individual elements of typed dictionaries (TypedDict) with the Required and NotRequired labels to determine required and non-required fields (by default, all declared fields are required to be filled in if the total parameter is not set to False).
• The TaskGroup class was added to the asyncio module with an implementation of an asynchronous context manager waiting for the task group to complete. Adding tasks to a group is done using the create_task () method.
• Added a decorator of classes, methods and functions @ dataclass_transform, when specifying which the static type checking system treats the object as when using the decorator @ dataclasses.dataclass. In the example below, the CustomerModel class, when validating types, will be processed by analogy with the class with the @ dataclasses.dataclass decorator, that is, as having the __ init __ method that allows id and name variables.
• In regular expressions, the ability to use atomic grouping ((? >...)) and jealous (possible) quantifiers (* +, + +,? +, {m, n} +) has been added.
• Added command line option "-P" and environment variable PYTHONSAFEPATH to disable automatic attachment of potentially unsafe file paths to sys.path.
• The py.exe utility is optimized for the Windows platform, which implements support for the syntax "-V <company><tag>:/" in addition to" -. "<major><minor>
• Many macros in the C API are converted to normal or static inline functions.
• Declared obsolete and will be removed in Python release 3.13 modules uu, cgi, pipes, crypt, aifc, chunk, msilib, telnetlib, audioop, nis, sndhdr, imghdr, nntplib, spwd, xdrlib, cgitb, mailcap, ossaudiodev and sunau. Removed functions PyUnicode_Encode *^[3].

A long-standing vulnerability in the Python tarfile module, affecting more than 350 thousand projects

It turned out that discovered in 2007 vulnerability under identifier the CVE-2007-4559 in the Python module tarfile affects more than 350,000 open source projects. source code This became known on September 22, 2022. The flaw is associated with the extract and extractall functions in the tarfile module, allows to hackers you to bypass attack the directory (path traversal) and overwrite arbitrary ones by files adding the sequence '..' to the file names in the tar archive.

Simply put, an attacker can exploit the flaw by downloading the tar archive in such a way that it is possible to exit the directory to which the file should be extracted, and then execute arbitrary code and gain control over the victim's device.

Illustration: securitylab.ru

Unusually, no one fixed the vulnerability back in 2007 - the issue was closed after updating the documentation. The update warned: 'Never extract archives from untrusted sources without first checking because files can be created outside the path'.

Trellix, whose specialists discovered the vulnerability, has created its own tool called Creosote, which helps you search for CVE-2007-4559. It was with its help that researchers found a vulnerability in the Spyder Python IDE and Polemarch.

In addition, Trellix experts have already prepared fixes for more than 11,000 projects. Researchers expect more than 70,000 repositories to receive fixes.

Additional technical details and examples of CVE-2007-4559 exploitation in the Windows version of the Spyder IDE and the Polemarch web interface can be found in the Trellix report^[4].

Introducing hacker software to Python libraries that steals keys to the Amazon cloud

On June 27, 2022, it became known that researchers discovered in the official third-party Python repository a set of malicious packages that can steal access keys to Amazon Web Services cloud resources and environment variables and output this data to an external public endpoint.

Sonatype expert Ax Sharma discovered five such packages: loglib-modules, pyg-modules, pygrata, pygrata-utils and hkg-sol-utils.

After analyzing these packages, two other Sonatype researchers, Jorge Cardona and Carlos Fernandez, found out that they contain code that reads and outputs confidential data itself or uses certain dependencies to carry out this operation.

Sharma points out that the first two packages apparently target users of the legitimate loglib and pyg libraries. Pygrata-utils contains code identical to loglib-modules. Who the other two packages are targeting is unknown.

All stolen data is placed without encryption in the form of.TXT files on a public endpoint (of what kind, not specified; it could be a server, it could be a PC). That is, in fact, they are in the public domain.

Aks Sharma wonders if this was done by mistake, or if the authors of malicious packages specifically share everything with demonstration purposes.

Information about malicious packets was transferred to PyPI, and they quickly disappeared from the repository. The endpoint on which the stolen data was unloaded also disappeared from the Web.^[5]

Breaking the Python package: the data of thousands of developers is at risk

On May 25, 2022, it became known that harmful the ctx version of the package allows you to hackers to get credentials. data AWS

Illustration: securitylab.ru

The incident was investigated by Sonatype, SANS Institute and an independent researcher. To the attack two libraries were subjected, but only one of them could cause serious damage. Ctx had 22,000 downloads a week and was hacked on May 14. The last update for the library was downloaded to the Python Package Index (PyPI) in December 2014. Having gained control of the library, malefactors they downloaded their versions - 0.1.2 (the latest version of the original), 0.2.2 and 0.2.6, which include functionality for thefts and data transfer to. servers hackers

One version was aimed at obtaining the AWS access key ID, computer name, and AWS secret access key when creating a dictionary. Another malicious version of ctx tried to get all the victim's environment variables.

The second compromised library was PHPass, a portable PHP system for hashing passwords. The original PHPass was removed in September 2021 along with the developer account, but the attackers were able to restore access to the project on GitHub.

On May 25, 2022, both hacked libraries were removed. Experts recommend that developers check the versions of recently downloaded ctx and PHPass packages and, if a malicious version is found, immediately remove it from the device.^[6]

2020: Second in the TIOBE rankings

The Python programming language managed to overtake Java in popularity and for the first time in its history took second place in the TIOBE rating. This became known on November 5, 2020. The rating itself has existed since 2003.

In the TIOBE ranking for November 2019, Java was in first place, and Python was content with third. Java's popularity fell by 4.57% over the year, while demand for Python increased by 2.27%. But while these languages ​ ​ were "arguing" for "silver," the victory in the current TIOBE rating was won by language C.

According to the ZDnet portal, the surge in Python's popularity may be associated with an increase in the pace of development of those areas where it is most often used. Examples included Data Mining, numerical calculations, and machine learning. TIOBE CEO Paul Jansen believes that interest in Python is also based on the fact that, if previously, mainly engineers were engaged in programming, now this skill "is required almost everywhere, and there are not enough good software developers."

The recent surge in Python's popularity is due to the fact that it is simple enough for non-programmers to use, and is not a language only for advanced programmers, said Paul Jansen.

If in the TIOBE rating for November 2020 Python occupied only the second line, then, according to employees of the Institute electronic engineers of Electrical and Electronics Engineers (IEEE), it has no equal. In July 2020, they published their own rating of the popularity of programming languages, where the first three consist of the same participants as in the TIOBE list, but they are located completely differently.

Rating of popularity of programming languages

In the IEEE ranking, Python holds the first place, and Java is in the second. Language C has to be content with Python "bronze"^[7].

Programming languages

• PHP
• JavaScript
• Java
• Python
• C++
• C#
• COBOL
• Bosque
• Project Verona
• CRN++
• Dart
• Kotlin
• Golang
• Rust
• Perl

Notes