Planet Python
Last update: May 30, 2021 04:40 PM UTC
May 30, 2021
Zero to Mastery
Python Monthly đ»đ May 2021
18th issue of Python Monthly! Read by 20,000+ Python developers every month. This monthly Python newsletter is focused on keeping you up to date with the industry and keeping your skills sharp, without wasting your valuable time.
PyPy
#pypy IRC moves to Libera.Chat
Following the example of many other FOSS projects, the PyPy team has
decided to move its official #pypy IRC channel from Freenode to
Libera.Chat: irc.libera.chat/hpy
The core devs will no longer be present on the Freenode channel, so we recommend to join the new channel as soon as possible.
wikimedia.org has a nice guide on how to setup your client to migrate from Freenode to Libera.Chat.
May 29, 2021
Python Software Foundation
The 2021 Python Language Summit
Every year, a small group of core developers from Python implementations such as CPython, PyPy, Jython, and more come together to share information, discuss problems, and seek consensus in order to help Python continue to flourish.
The Python Language Summit features short presentations followed by group discussions. The topics can relate to the language itself, the standard library, the development process, documentation, packaging, and more! In 2021, the summit was held over two days by videoconference and was led by Mariatta Wijaya and Ćukasz Langa.
If you weren't able to attend the summit, then you can still stay up to date with what's happening in the world of Python by reading blog posts about all of the talks that were given. Over the next few weeks, you'll be able to dive into all of the news from the summit so you can join in on the big conversations that are happening in the Python community.
Day 1
Day 2
The 2021 Python Language Summit: What Is the stdlib?
Brett Cannon gave a presentation at the 2021 Python Language Summit about the standard library in order to start a conversation about whether it's time to write a PEP that more clearly defines it.
What Is the stdlib?
He succinctly described the stdlib as "a collection of modules that ship with CPython (usually)." This was the most accurate definition he could give, considering how big it is, how varied its contents are, and how long it has been around.
He didn't offer an answer to the question about whether or not there should be a new informational PEP to define clear goals for the stblib, but he wanted core developers to engage with the question. There are a variety of opinions on the stdlib, but it could be beneficial to come to some kind of agreement about:
- What it should be
- How to manage it
- What it should focus on
- How to decide what will or will not be added to it
He shared that he semi-regularly sees requests for adding a TOML parser to the stdlib. When he considers requests, he asks himself:
- Should a module be added?
- How should such a decision be made?
- What API should it have?
- Is there a limit to how big the stdlib should get?
So far, there haven't been basic guidelines for answering these kind of questions. As a result, decisions have been made on a case-by-case basis.
How Big Is the stdlib?
He did some data digging in March of 2021 and shared his findings. Here are the broad strokes:
- python -v -S -c pass imports 14 modules.
- The are 208 top-level modules in the standard library.
- The ratio of modules to people registered to vote in the last Steering Council election is 2.3, but not all of those people are equally available to help maintain the stdlib.
What Should the stdlib Cover?
Some people have suggested that the stdlib should be focused on helping users bootstrap pip and no more. Others have said that it should focus on system administration or other areas. Considering that there are thirty-one thematic groupings in the index, the people who maintain the stdlib don't seem to have come to a collective decision either. The groupings cover everything from networking to GUI libraries to REPLs and more.
The stdlib has been around for a long time, and we need to be careful about breaking people's code, so the goal is not to deprecate what is already there but to consider guidelines for making additions.
How Do We Decide What Goes Into the stdlib?
He compared PEP 603 and graphlib to show how this question has been answered in different ways in the past. The goal of PEP 603 was to add the class frozenmap to the collections module. However, the graphlib module was only ever discussed in an issue and never got a PEP before it was added to the stdlib. There is no standardized approach for making these kinds of decisions, so he would like to know what approaches core developers think would be most appropriate.
What Is the Maintenance Cost?
The PR queue is already long, which can be overwhelming for maintainers and discouraging for contributors.
The following modules aren't used in any of the top 4000 projects:
- mailcap
- binhex
- chunk
- nis
Seventy-six modules are used in less than 1% of the 4000 most downloaded projects on PyPI. That's over 36% of all the modules in the stdlib. This raises some questions:
- Do we want to continue to ship these modules?
- What does this tell us about what the community finds useful in the stdlib?
- How can that inform future guidelines about what to include in the stdlib?
Based on the data from March 2021, there were:
- 37 modules with no open PRs
- 1,451 PRs involving the stdlib, which made up the bulk of all the PRs
Weekly Python StackOverflow Report
(cclxxvii) stackoverflow python report
These are the ten most rated questions at Stack Overflow last week.
Between brackets: [question score / answers count]
Build date: 2021-05-29 18:33:34 GMT
- Remove any empty list present in the list - [10/7]
- Numpy empty list type inference - [7/2]
- ModuleNotFoundError: No module named 'grp' on windows - [6/1]
- How to configure fields in Django forms.Form? - [6/1]
- Vectorization or efficient way to calculate Longest Increasing subsequence of tuples with Pandas - [6/1]
- Python pandas: fast way to flatten JSON into rows by a surrogate key - [6/1]
- Convert a dataframe to a list of tuples - [5/4]
- How to build a heatmap? - [5/2]
- "No message found for this issue" when running pipeline with SonarQube - [5/0]
- Is there a way to use the previous calculated row value with the sum of a different column in a Pandas Dataframe? - [4/4]
The No TitleÂź Tech Blog
Just updated - both Optimize Images and Optimize Images X
This time, we are releasing both Optimize Images and Optimize Images X at the same time. The original CLI version now uses temporary files with in-memory buffers, which prevents unnecessary I/O, and also displays more detailed and more useful version information. Optimize Images X, the GUI version, is now compatible with Python3.7+ and has the ability to open or preview a selected image.
AI Pool
Introduction of Fast Fourier Transformation (FFT)
This article comprises of introduction to the Fourier series, Fourier analysis, Fourier transformation, why do we use it, an explanation of the FFT algorithm, and its implementation.
How to freeze tensorflow graph partly?
Couldn't find a way to freeze the TensorFlow graph partly. I just need to freeze a part of the weights of the network....
How to convert numbers to one hot vectors?
I want to train a Keras model and I use NumPy to convert numbers to a one-hot vector. why Keras does not provide the implementation inside the training process?...
softmax_cross_entropy_with_logits issue with keras
I'm using Keras with TensorFlow backend and using cross-entropy as the loss function. I saw that TensorFlow requires pre softmax as an input for the loss, but I put the output of the softmax in Keras . What is the right way to use it?...
Give some suggestions to avoid overfitting
I'm training a neural network that basically does binary classification, but it overfits too fast. After 4-5 epochs it's already in overfitting. What is the reason for it? How to avoid fast overfitting?...
Is there a way to visualize Keras model?
I'm designing a neural network model and would like to plot the architecture with weights and shapes. How to do that?...
Python Pool
Learn About Caesar Cipher in Python
Cryptography is the study of the science behind securely transmitting a message from a sender to a receiver. The goal is to prevent a third party from accessing a message. It is achieved by converting the original message into a form that can only be interpreted if one has access to the key. Thus, the key is known only to the sender and the receiver. There are several algorithms in cryptography, but today, we will be looking into caesar cipher in python.
Introduction to Caesar cipher Algorithm
Caesar cipher is one of the oldest and the most commonly known cryptography technique. It is a weak encryption technique for practical implementation but for learning the basics of cryptography. Therefore, it is the best one to get started with. It is based on substitution cipher. Each character is shifted by a specific number of characters, which is the key. Before we dive further into this algorithm, we will understand some basic cryptography terms.
Terms in cryptography
Plaintext: It is the original message which is sent by the sender to the receiver.
Ciphertext: It is obtained when an encryption algorithm has been applied on the plaintext. Basically, it is the scrambled form of the plaintext which cannot be interpreted without the key.
Encryption: It is the technique behind conversion of a plaintext to its ciphertext.
Decryption: It is the technique behind the conversion of the ciphertext to its original plaintext.
Key: It is the value that, when fed to the encryption and decryption algorithm, gives ciphertext and plaintext, respectively.
How does Caesar Cipher work in Python?
The working of the caesar cipher is dependent on the key selected. Each alphabet in the plaintext will be shifted by an amount equal to the value of the key.
For example, if our plaintext is: âABCDâ and the key is 1, then each letter will be replaced by the letter shifted by 1. So, A will be replaced by B, B will be replaced by C, C will be replaced by D, and D will be replaced by E. Thus, the ciphertext obtained would be âBCDE.â
Had the key size been 3, the ciphertext of the plaintext âABCDâ would be âDEFGâ.
Caesar Cipher Formula
The formula to convert a given plaintext âPâ to ciphertext âCâ using key âKâ is:
C = ( P + K ) % 26
Similarly, the formula to convert a given ciphertext âCâ to plaintext âPâ using key âKâ is:
P = ( C - K ) % 26
Here, we assign each alphabet a number â A = 0, B = 1, C = 2, D = 3,âŠ,Z = 25. We perform a summation between the number concerning the alphabet and the key value. Then, we perform a modulus operation on that value. For the obtained value, we assign it the alphabet, which is the ciphertext alphabet.
Suppose our plaintext was âATTACKâ and the key was 10, then for each letter, the encryption would be:
A : C1 = ( 0 + 10 ) % 26 = 10 % 26 = 10 = K
T : C2 = ( 19 + 10 ) % 26 = 29 % 26 = 3 = D
T : C3 = ( 19 + 10 ) % 26 = 29 % 26 = 3 = D
A : C4 = ( 0 + 10 ) % 26 = 10 % 26 = 10 = K
C : C5 = ( 2 + 10 ) % 26 = 12 % 26 = M
K : C6 = ( 10 + 10 ) % 26 = 20 % 26 = U
Therefore, the obtained ciphertext is : âKDDKMUâ
Caesar Cipher Using Python
Now, we will be writing the code for implementing the caesar cipher algorithm. We shall be defining two functions â one for encryption and another for decryption. Lets us look at both separately.
Encryption algorithm explaination
To perform encryption, we will be creating a user defined function â
caesar_encryption(): The function will accept two arguments â plaintext and the key-and print the encrypted ciphertext. We shall take the plaintext and the key as the input from the user and pass them into the function.
Inside the function, we have a string named encryption_str of zero length. We have taken a for loop for iterating the plaintext. Inside the for loop, we have taken three if-else conditions. The first one is for handling uppercase letters, the second one for lowercase letters, and the last else case is for handling non-alphabet characters. For checking uppercase and lowercase, we have used isupper(), and islower() functions.
Then we have used the ord() function to obtain the Unicode code point representation of the letter. The uppercase alphabets start from the Unicode value 65 with A = 65, B = 66, etc. Similarly, the lowercase alphabets start from 97 with a = 97, b = 98, etc. We have subtracted 65 and 97 for uppercase and lowercase characters respectively to reduce each alphabet to the A = 0, B = 1, C = 2,âŠ,Z = 25 representation. And then, we have applied the âC = ( P + K ) % 26â formula and added 65 for lowercase and 97 for uppercase for that value to get the Unicode integer value for the obtained ciphertext letter.
Then, we have to use the chr() function, which is used to obtain the alphabet from its Unicode integer value. We have simply concatenated that character with the encryption_str string. If the character is not an alphabet, we would simply concatenate it as it is with the encryption_str string.
def caesar_encryption(plaintext,key):
encryption_str = ''
for i in plaintext:
if i.isupper():
temp = 65 + ((ord(i) - 65 + key) % 26)
encryption_str = encryption_str + chr(temp)
elif i.islower():
temp = 97 + ((ord(i) - 97 + key) % 26)
encryption_str = encryption_str + chr(temp)
else:
encryption_str = encryption_str + i
print("The ciphertext is:",encryption_str)
plaintext = input("Enter the plaintext:")
key = int(input("Enter the key:"))
caesar_encryption(plaintext,key)
The output for the above encryption algorithm is:
Enter the plaintext:attack on the palace! Enter the key:10 The ciphertext is: kddkmu yx dro zkvkmo!
Decryption algorithm explaination
We are using the same method for performing decryption. We have a user-defined function name caesar_decryption() which takes the ciphertext and the key as the arguments and prints the plaintext. It takes the ciphertext and key as input from the users.
We have a zero-length string named decryption_str, with which we will be concatenating our plaintext. Apart from the formula, the only slight difference here is that for both uppercase and lowercase if statements, we will be checking if the ( C â K ) value is negative or not from the âP = ( C â K ) % 26â formula. If it is negative, then we will add 26 to it before performing modulus. If not, then we will refrain from doing that. Just like the encryption algorithm, the no alphabet characters will be unchanged.
def caesar_decryption(ciphertext,key):
decryption_str = ''
for i in ciphertext:
if i.isupper():
if ((ord(i) - 65 - key) < 0):
temp = 65 + ((ord(i) - 65 - key + 26) % 26)
else:
temp = 65 + ((ord(i) - 65 - key) % 26)
decryption_str = decryption_str + chr(temp)
elif i.islower():
if ((ord(i) - 97 - key) < 0):
temp = 97 + ((ord(i) - 97 - key + 26) % 26)
else:
temp = 97 + ((ord(i) - 97 - key) % 26)
decryption_str = decryption_str + chr(temp)
else:
decryption_str = decryption_str + i
print("The plaintext is:",decryption_str)
ciphertext = input("Enter the ciphertext:")
key = int(input("Enter the key:"))
caesar_decryption(ciphertext,key)
We will be taking the same ciphertext from the encryption algorithm example to check the plaintext. The output will be:
Enter the ciphertext:kddkmu yx dro zkvkmo! Enter the key:10 The plaintext is: attack on the palace!
FAQâs
The limitation of the caesar cipher is that it is prone to brute force attack, meaning that by trying every key combination, the plaintext can be obtained. This is because there are only 26 unique keys possible.
That wraps up caesar cipher with python. Do let us know your views in the comments below.
Till then, Keep Learning!
The post Learn About Caesar Cipher in Python appeared first on Python Pool.
May 28, 2021
Test and Code
155: Four Questions to Ask Frequently During Software Projects - Tim Ottinger
Tim Ottinger has four questions that work great in many situations, from doing homework, to cooking, to writing code, to entire software projects.
They are actually awesome questions to ask during a software project.
We discuss the questions, where they came from, and look at some uses in software.
The questions:
- What is it that needs to be done?
- What do we need in order to do it?
- Where can we get what we need?
- How can we tell if weâre doing it right?
Bonus question that can be swapped out for #1:
- What's the most important thing that it doesn't do yet?
Special Guest: Tim Ottinger.
Sponsored By:
- PyCharm Professional: Try PyCharm Pro for 4 months and learn how PyCharm will save you time. Promo Code: TESTANDCODE21
- ConfigCat.com: Release features faster with less risk with ConfigCat. You can try it out with the forever free plan or get 25% off with code testandcode2021 Promo Code: testandcode2021
Support Test & Code : Python Testing
Links:
<p>Tim Ottinger has four questions that work great in many situations, from doing homework, to cooking, to writing code, to entire software projects.</p> <p>They are actually awesome questions to ask during a software project.</p> <p>We discuss the questions, where they came from, and look at some uses in software.</p> <p>The questions:</p> <ol> <li>What is it that needs to be done?</li> <li>What do we need in order to do it?</li> <li>Where can we get what we need?</li> <li>How can we tell if weâre doing it right?</li> </ol> <p>Bonus question that can be swapped out for #1:</p> <ol> <li>What's the most important thing that it doesn't do yet?</li> </ol><p>Special Guest: Tim Ottinger.</p><p>Sponsored By:</p><ul><li><a href="https://testandcode.com/pycharm" rel="nofollow">PyCharm Professional</a>: <a href="https://testandcode.com/pycharm" rel="nofollow">Try PyCharm Pro for 4 months and learn how PyCharm will save you time.</a> Promo Code: TESTANDCODE21</li><li><a href="https://configcat.com/" rel="nofollow">ConfigCat.com</a>: <a href="https://configcat.com/" rel="nofollow">Release features faster with less risk with ConfigCat. You can try it out with the forever free plan or get 25% off with code testandcode2021</a> Promo Code: testandcode2021</li></ul><p><a href="https://www.patreon.com/testpodcast" rel="payment">Support Test & Code : Python Testing</a></p><p>Links:</p><ul><li><a href="https://www.industriallogic.com/blog/the-four-questions/" title="The Four Questions" rel="nofollow">The Four Questions</a></li></ul>Python Morsels
What is a callable?
Transcript
A callable is an object that you can call.
Functions are callable objects
When you define a function in Python:
>>> def greet(name):
... print("Hi", name)
...
You'll get a function object:
>>> greet
<function greet at 0x7f61693c5940>
You can call a function object by putting parentheses (()) after it:
>>> greet("Trey")
Hi Trey
So, functions are callables, meaning they're objects that you're able to call.
Is it a function or a class?
Functions are not the only callables in Python.
If we call the built-in enumerate function, we will get back an enumerate object:
>>> colors = ["green", "purple", "white", "pink"]
>>> enumerate(colors)
<enumerate object at 0x7f816c0ad400>
We can loop over that enumerate object in order to see what's in it (related: looping with indexes):
>>> list(enumerate(colors))
[(0, 'green'), (1, 'purple'), (2, 'white'), (3, 'pink')]
Looping is the one thing we can do with an enumerate object.
The reason we get back an enumerate object when we call the enumerate function, is that enumerate isn't actually a function: it's a class!
>>> enumerate
<class 'enumerate'>
We call enumerate a function because we often use the word "function" in a fuzzy way in Python.
In Python we think in terms of duck typing. So if something is a function-like object (meaning it acts like a function) we may simply call it "a function".
The built-in enumerate function acts like a function (you call it using parenthesis) which means it's basically a function.
But more properly, enumerate is a callable, meaning it's an object that can be called.
In Python, functions can be called and classes can be called.
Callable class instances
Functions and classes are both callables, but you can actually invent your own callables too.
We have a class here called Person:
class Person:
def __init__(self, name):
self.name = name
def __call__(self):
print("My name is", self.name)
When we call the Person class:
>>> trey
<__main__.Person object at 0x7fbf9f3331c0>
We'll get back an instance of that class (a Person object):
>>> trey = Person("Trey")
We can also call that Person object by putting parentheses after it:
>>> trey()
My name is Trey
This works because we've implemented a __call__ method on the Person class.
Adding a __call__ to a class makes its class instances callable.
Summary
A callable is a function-like object, meaning it's something that behaves like a function. You can put parentheses after a reference to a callable to call it.
Functions are callables and classes are callables. They're not the only callables, but they're the primary two types of callables that you'll see in Python.
PyCharm
Early Access PyCharm: PyCharm and the Web
In this episode, we talk to Andrey from the Webstorm team, and find out how the web has changed over his time at the Webstorm. We dive into how itâs like leading a product like Webstorm to how things are different from the jQuery days. We talk about how PyCharm Pro now supports everything from jQuery to React as well as languages like TypeScript and even CoffeeScript. Lastly, we talk about what the team is thinking about and working on now.
Transcript
Nafiul: Welcome to Early Access PyCharm. Iâm your host Nafiul Islam.
Today, weâre going to be talking to Andrey from the WebStorm team. And heâs going to tell us a little bit about how PyCharmâs web capabilities are built and what has changed over the course of his tenure as a member of the WebStorm team and what theyâve got planned for the future.
So Andrey, the only thing that I know about you is that you lead the WebStorm team. What does WebStorm have to do with PyCharm?
Andrey Starovoyt: There are two questions here. First of all, I want to mention actually, I am the team lead of the Webstorm team since the last year, a bit more. Before it was another team lead. So for me itâs some kind of new experience and it was so really interesting and challenging because it was the time of the lockdown.
Of course, we have a remote or distributed team, because some colleagues work in Munich, some in Poland. And also we have developers in Amsterdam and St. Petersburg.
Nafiul: Makes sense. So this is a new experience for you. WebStorm is actually our web offering. This is what we use to build our front-end technologies. Um, tell me a little bit more about how WebStorm came to be. And essentially where it is right now.
Andrey Starovoyt: Itâs an interesting question, because, back then, WebStorm was a part of PhpStorm, as far as I know. And at some point that was the decision to split the team into two.
Like a PhpStorm team and WebStorm team, that will handle everything related to JavaScript development. And I believe it was a good decision, because now JavaScript is not only a front-end language or language for front-end development, but it is also a back-end language.
Nafiul: Yes, we have NodeJS. And, uh, also Deno. Do we support Deno yet in WebStorm?
Andrey Starovoyt: Yes, we have a blog and from Deno.
Nafiul: Oh, awesome.
Andrey Starovoyt: Yeah.
After that decision we started to work more precisely on JavaScript technology development. Some people said that it was really challenging back then, because when I joined the Webstorm team, front-end started to grow very fast. Like, we had a new framework every week and it was all about decisions.
Like, do we really need to support this framework in our IDE or it will die in a couple of months. If you talk about WebStorm at this moment or right now â itâs a different thing because now we have three major frameworks: Angular, Vue and React. Of course, some people can say thatâs there is also Svelte. But, itâs not so popular yet. And everything that we considered to develop or to improve is related to these technologies and of course, to JavaScript language itself.
Nafiul: So tell me about the contrast between whatâs happening now. Because right now you have to support Vue, React, and Angular.
And there are multiple versions of Angular and they come up with a new one all the time. So tell me a little bit about, you know, how it was like in the good old days of jQuery, when jQuery was the web development and now itâs not.
Andrey Starovoyt: Actually, Iâm not sure. It was in jQuery times. Uh, I know how it was when I joined the team, it was.
Nafiul: When did you join the WebStorm team?
Andrey Starovoyt: Seven years ago.
Nafiul: Seven years ago. My goodness. Youâre an O G.
Andrey Starovoyt: Yeah. The problem there was, there are too many frameworks. In 2014, ES6 standard wasnât yet released. So we had some proposals, we had some features that can be a part of the standard. But, we werenât sure about it.
And, there are a lot of languages, similar to Javascript. For example, CoffeeScript. Do you know CoffeeScript?
Nafiul: I personally used to use something called IcedCoffeeScript. It had something, it had two keywords async and defer. So it made handling callbacks a lot easier, but this is like ancient history.
Andrey Starovoyt: Yes. Yes. And itâs an interesting fact that CoffeeScript had several dialects, including Ice Script. And it was the same for all the parts, for the front-end development.
Nafiul: So back then, you know, everything was emerging. Everything from languages, dialects of languages, frameworks⊠I mean, do you remember CanJS and Ember.js and Backbone.js and all these different frameworks? Why do you think people kind of settled on React?
Andrey Starovoyt: Um, itâs an interesting question. Uh, one of the reasons, I believe, is that it was supported by huge companies like Facebook. And also each introduced a good concept that was really easy to understand and to implement. And letâs say, I think we can say about Angular, like you turned to use two very simple conceptions and it is also supported by a huge company. Uh, and it follows the reason why around that framework because the technology can be built by the good community.
But it doesnât work all the time, but, I believe it worked for Angular and React.
Nafiul: I see. So, what I found interesting about the whole React and Angular thing is that it was a move towards single-page applications. But that meant like typical frameworks, such as Django in the Python world, Flask in the Python world⊠You know, they were mostly making APIs.
So you saw a rise of back-ends, just making APIs that were simple layers over the databases, right? They provided authentication. They provided rate, limiting tags, all that good stuff. And the front-end, it still had the NodeJS server because you still need to serve these files. Right. But you were working with single-page applications.
So why is it that things are slowly coming around back to server-side rendering, after all this time?
Andrey Starovoyt: Oh, actually, I donât know. We can guess that, or just very simple for front-end developers or to understand how it works and you can focus on the technologies or you donât need to think about the back-end at all.
Itâs just an API for you. And you can build ever since that you need to own the front-end site.
Nafiul: Makes sense. But you know, now that server-side rendering is slowly coming back. I think thereâs hot wire. I think, React also made some leeways into having server-side rendering. Does it feel like itâs just whatâs in fashion or is it that? There are benefits and people are like, you know what, maybe single-page applications involve just too much work.
Andrey Starovoyt: Right now it works, the ways that you use server-side rendering and still use all single-page application conceptions. So we just a way to speed up the rendering at the first time. And after that you can work with a rich application.
Nafiul: Youâre kind of getting the best of both worlds.
Andrey Starovoyt: Yeah, definitely. If I talk about server-side rendering for front-end development, itâs usually done also on the NodeJS side. And you donât need to invest time in studying other technologies. So you can do everything using JavaScript and React, for example.
And it just will work.
Nafiul: Makes sense, but you know⊠One of my pet peeves about frontend in general is that itâs very complicated. And I know thatâs a subjective way of looking at it, but you know, you have pre-compiled languages like TypeScript that have their own map files. And then you have this thing called webpack, which is a monster in and of itself.
How do you keep up with all these tools that are popping up every five seconds? Because the front-end isnât simple anymore. Back in the good old days⊠Well, I think itâs the good old days⊠You just put a script tag in your HTML, write some jQuery and thatâs it. Now you have a pre-compiler.
You have PostCSS, you have this, you have that. How on earth do you get the time to actually make tooling for all this?
Andrey Starovoyt: Actually, if we talk about onboarding⊠Like you want to build an application, you can just use âCreate React appâ and it just works. So you donât need to think about all the details, like how webpack works or React translated into the JavaScript code and so on. And actually, I believe the same problems there are with all peak languages, like letâs call it peak. If we talk about Java, there is Spring Boot, that is just a framework that solves almost every possible problem about web development.
And you cannot just understand how it works. Itâs magic. If you start to learn to work, if you start to use it, itâs magic for you. And itâs the same tool for all frameworks. If we talk about Java, we can also mention Hibernate or something like that.
Nafiul: So basically youâre defending the complication of everything. Itâs like, Hey, we do everything. So itâs okay. If it does everything.
Andrey Starovoyt: But we understand why we do it, why we need webpack, why we need TypeScript. Because it solves some problems and here they are set up in general is a part of magic because you donât understand how webpack works. But, uh, you know, that webpack builds for you to optimize it, to call towards that includes everything that you need and so on.
And yeah, it helps.
Nafiul: Yeah. In webpack we trust. So, um, youâve been building new features in WebStorm like crazy. Iâve been looking at your release notes. A lot of these things are coming over to PyCharm as well. Tell me about, you know, what are you going to be focused on next? What is your next big thing for WebStorm?
Andrey Starovoyt: As I mentioned before, we donât have a lot of frameworks that we need to support, because JavaScript development is not splitted, like it was before. Right now we have three major frameworks like Angular or React or Vue. And we want to improve the overall experience for the users of the frameworks, like introducing new inspections, introducing new fixes, introducing integrations for the frameworks. Itâs one of our focuses. Of course, we need to support new versions of languages including TypeScript and JavaScript, something that has to be done by default and also we think a lot about onboarding. We have a lot of features, a lot of refactoring fixes and so on. And for a lot of these features a lot of our users just donât know about the features. Very sad for us.
Nafiul: We have the same problem in PyCharm. We do a lot, but not all the features are discoverable. So on that note thank you so much for joining us and weâll hope to see you again soon.
Andrey Starovoyt: See you soon. Bye.
Real Python
The Real Python Podcast â Episode #62: Selecting the Ideal Data Structure & Unravelling Python's "pass" and "with"
How do you know you're using the correct data structure for your Python project? There are so many built into Python and even more that are importable from the collections module. This week on the show, David Amos is back, and he's brought another batch of PyCoder's Weekly articles and projects. We discuss a recent three-part video course on selecting the ideal data structure.
[ Improve Your Python With đ Python Tricks đ â Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Python Pool
Python Check if File Exists â All Methods Covered
We can use python to create different types of files and store them in the local computer storage. Mainly there are two types of files â binary file and text file. We can perform operations such as reading, appending, writing, and creating. While handling a given file, we want to perform certain operations only if the file exists to avoid unwanted errors. So, python must check if a file exists. There are several methods for that, and we shall be looking into these methods.
The need and types of ways to check if a file exists
We want to check if a file exists or not to avoid unwanted errors. It is important to check so as to prevent overwriting a given file. There are three main ways to check if a file exists or not. Firstly, we can check by using exception handling. The second method is by using the os module and the third way is by using the pathlib module.
Python check if file exists using Exception Handling
To check if a file exists, we can use exception handling. It is one of the simplest ways to check if a file exists. For that, we use the open() function. There are two types of exceptions which can be thrown :
- IOError
- FileNotFoundError
IOError
First, we will look into IOError. IOError is raised when an input operation or an output operation fails. In this case, it fails because open() function was trying to access a file which does not exist. We will have a try-except block. In the try block, we shall try to open a file and when the file is not found, it will raise IOError from the except block.
try:
f = open("filename.txt")
f.close()
except IOError:
print("File not found. IOError occured.")
The output is shown below:
File not found. IOError occured.
Since no file named âfilename.txtâ existed in the system, it showed IOError.
FileNotFoundError
Another exception which we can throw is FileNotFoundError. The FileNotFoundError is raised when a file which we are trying to access does not exist in the system.
try:
f = open("filename.txt")
f.close()
except FileNotFoundError:
print("File not found. FileNotFoundError occured.")
Here the output will print the print statement from the exception since the file âfilename.txtâ does not exist.
File not found. FileNotFoundError occured.
Python check if file exists using the OS module
The OS module present in python contains functions that are used for interacting with the operating system of the computer. There are three functions in the os module which are used for checking the existence of the file. The isfile(), isdir() and exists() function from the os.path module is used for the same. We shall be looking into each of the functions individually.
os.path.isfile()
The os.path.isfile() takes the path of the file, which has to be checked as an argument. It returns a bool value that will be true if the path exists and false if it does not exist.
First, we will have to import the os module. Then we will assign the variable âpath_nameâ the path of the file. Since the below code has been executed in the colab, the path name is the filename. Finally, we will call the function os.path.exists() and pass the variable âpath_nameâ as the argument to the function.
If the file is found at the specified location, the function will return the true value, and it will print that the file exists. We can also include any further file operations which have to be executed if the file exists, in the first if condition. If the function returns a false value, it means that the file does not exist. So, it will print the second statement.
import os
path_name = "filename.txt"
if os.path.exists(path_name):
print("File exists")
else:
print("File does not exist")
The output to the above code if the âfilename.txtâ file does not exist is:
File does not exist
os.path.isdir()
The function os.path.isdir() checks a given directory to see if the file is present or not. It accepts the directoryâs path as the argument and returns a boolean value depending on whether the directory exists. We have a folder in colab saved as âMyDirectoryâ. We will use os.path.isdir() to check whether that directory exists or not.
First, we will import the os module. Then, we shall specify the path_name as the name of the directory and use an if-else case to check whether that directory exists or not.
import os
path_name = "MyDirectory"
if os.path.isdir(path_name):
print("File exists")
else:
print("File does not exist")
Since the folder exists, it will print âFile existsâ.
File exists
But, if we delete the folder, it will print the else statement.
File does not exist
os.path.exists()
The function os.path.exists() checks if the mentioned path exists or not. Just like the other two functions, it accepts the path as an argument and returns a boolean value.
Here, in the google colab, we have two folders â a MainFolder and a SubFolder located inside the MainFolder.
So, we will use os.path.exists() to check if the path for SubFolder exists or not.
import os
path_name = "/content/MainFolder/SubFolder"
if os.path.exists(path_name):
print("File exists")
else:
print("File does not exist")
Since the file exists, it will print âFile existsâ.
File exists
Python check if file exists using the Pathlib module
The pathlib module in python is used to interact with the filesystem. There are three functions present in the pathlib module â exists(), is_file() and is_dir(). We will try all these three methods to see how they work.
pathlib.Path.exists()
This function exists() from the pathlib module checks whether a given path exists or not. First, we will import the pathlib module. Then, we shall pass the path_name as an argument to the Path() function, which shall return a value. Using that, we will call the exists() method. Here, instead of using an if-else case, we will print the functionâs return value.
import pathlib
path_name = "MyDirectory"
p = Path(path_name)
print(p.exists())
Since the path exists, the output will be True.
True
pathlib.Path.is_dir()
We will check if a given directory exists or not using the is_dir() function. We will pass the directory name as an argument to the Path() function and then with the return value, we shall call the is_dir() function.
import pathlib
path_name = "/content/MyDirectory"
p = Path(path_name)
print(p.is_dir())
The output for an existing directory shall be:
True
pathlib.Path.is_file()
The is_file() function shall check if the given file exists or not. Next, we shall pass the file name to the Path() function. It will return a value with which we will call the is_file() function and print is_file()âs return value.
import pathlib
path_name = "filename.txt"
p = Path(path_name)
print(p.is_file())
Since filename.txt exists, it will print True.
True
If the file would not be found, it would print False.
- You might also be interested in reading ăHow toă Check if the Variable Exists in Python.
FAQâs on python check if file exists
The ideal method depends on your requirement. For example, if you want to check particularly for a given file, you can use the exists() method from the os and pathlib module and also through the open() method in exception handling. The other methods will check for a given directory or a given path.
That was all folks! If you have anything to share, feel free to let us know in the comments.
Till then, Keep Learning!
The post Python Check if File Exists â All Methods Covered appeared first on Python Pool.
Talk Python to Me
#318: Measuring your ML impact with CodeCarbon
Machine learning has made huge advancements in the past couple of years. We now have ML models helping doctors catch disease early. Google is using ML to suggest traffic routes in their maps app that will lesson the amount of gasoline used in the trip. And many more examples. But there is a heavy cost for training machine learning models. <br/> <br/> In this episode, you'll meet Victor Schmidt, Jonathan Wilson, and Boris Feld. They work on the CodeCarbon project together. This project offers a Python package and dashboarding tool to help you understand and minimize your ML model's environmental impact.<br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Guests</b><br/> <b>Victor Schmidt</b>: <a href="https://twitter.com/vict0rsch" target="_blank" rel="noopener">@vict0rsch</a><br/> <b>Jonathan Wilson</b>: <a href="https://www.haverford.edu/users/jwilson" target="_blank" rel="noopener">haverford.edu</a><br/> <b>Boris Feld</b>: <a href="https://twitter.com/Lothiraldan" target="_blank" rel="noopener">@Lothiraldan</a><br/> <br/> <b>CodeCarbon project</b>: <a href="https://codecarbon.io/" target="_blank" rel="noopener">codecarbon.io</a><br/> <b>MIT "5 Cars" Article</b>: <a href="https://www.technologyreview.com/2019/06/06/239031/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/" target="_blank" rel="noopener">technologyreview.com</a><br/> <br/> <b>Green Future Choice Renewable Power in Portland</b>: <a href="https://portlandgeneral.com/energy-choices/renewable-power/green-future-choice" target="_blank" rel="noopener">portlandgeneral.com</a><br/> <b>YouTube Live Stream</b>: <a href="https://www.youtube.com/watch?v=cvYUosFIqpk" target="_blank" rel="noopener">youtube.com</a><br/></div><br/> <strong>Sponsors</strong><br/> <br/> <a href='https://talkpython.fm/square'>Square</a><br> <a href='https://talkpython.fm/training'>Talk Python Training</a>
Abhijeet Pal
Find Missing Number in a given Array Using Python
Problem Definition Find the missing numbers in a given list or array using Python. For example in the arr = [1,2,4,5] the integer â3â is the missing number. There are ... Read more
The post Find Missing Number in a given Array Using Python appeared first on Django Central.
Python Pool
PIP vs PIP3: What is the Difference?
Hey geeks! Let us start our discussion on todayâs topic. If you are a python developer, you must have understood our agenda after reading the title. But if you are a beginner to python, then no worries; today, we will have our detailed discussion on what PIP and PIP3 are, when and how to use PIP and PIP3, the PIP vs PIP3 comparison, and a lot more.
So if you want to understand PIP and PIP3, then you are in the right place. So, letâs get started.
What is PIP in python?
Pip is the standard package manager for Python. It allows us to install and manage additional packages that are not part of the Python standard library.
Package management is so important that pip has been included with the Python installer since versions 3.4 for Python 3 and 2.7.9 for Python 2, and itâs used by many Python projects, which makes it an essential tool for every Pythonista.
The concept of a package manager might be familiar to you if you are coming from other languages. JavaScript uses npm for package management, Ruby uses gem, and .NET uses NuGet. In Python, pip has become the standard package manager.
To check the version of pip
python -m pip version
pip 21.1.1 from C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\site-packages\pip (python 3.9)
You should see a similar output displaying the pip version, the location, and version of Python.
To learn in detail how to check the python version in different OS, click here.
Why install pip3 instead of using easy_install?
Itâs definitely worth your time to install pip3. Remember: PIP was designed to be a replacement for easy_install and to solve many of the problems it caused.
Here are some reasons you should definitely install pip3:
- Because all packages are downloaded before installation, you never run into the problem of installations being only partially completed.
- When you install pip3, it is easy to keep track of why you wanted to install a package in the first place because PIP keeps track of that information.
- The error messages speed up the learning process and make it easier to figure out why things might have gone wrong.
- Packages can be installed flat and donât need to be installed as egg packages.
- With pip3, itâs easier to define fixed requirements, so you can reliably produce packages.
- The distribution package that included easy_install is no longer maintained. Instead, it was absorbed into setup tools. So, if you try to install distribute, youâll only end up installing setup tools instead.
When and How to Use PIP and PIP3 in Python?
You can find the packages to download and install in the PyPI (Python Package Index) repository. Still, you can set up your repositories and install packages from them using the standard PIP tool. Moreover, you can create your package, publish it on PyPI, and make it available for installation by all Python users.
Typically, in manuals and instructions, package installation is written as:
pip install package_name
However, you may also see other options, such as pip3 install instead of pip install. So how do you correctly install packages on your system, and what is the difference between PIP and PIP3?
When you type pip in the console, the system looks for an executable file with that name in the current folder and then in the folders specified in the system PATH variable.
If you have multiple Python installations and all of them are in your PATH, you cannot be sure which directory will be searched first. This is because the Python installation directory has duplicate executables whose names contain the Python version.
For example, for the installed Python 3.9, these will be pip, pip3, and pip3.9. So, you can install the necessary modules for the desired version of Python.
Basically, if you type pip install package_name, you will get exactly the output you intended.
PIP Vs PIP3 Whatâs the Difference?
| PIP | PIP3 |
| PIP is a soft link for a particular installer. | pip3 is an updated version of pip which is used basically for python 3+. |
| The system will use one of your Python versions depending on what exactly is first in the system PATH variable. | When you run PIP3, you can be sure that the module will be installed in Python 3. |
| if I use pip, the package will be installed for the python version that I installed last on my computer. That would be 3.9 for my case. So using pip and using pip3.9 will produce the same result for me. | If I use pip3, the package will only be installed for python3. It wonât be available for 2.7 or 2.9. |
To install packages in python3, should we use pip or pip3?
Pip3 is the Python3 version of pip.
If you use pip, then only the python2.7 version will be installed. You have to use pip3 for it to be installed on Python3. So to install packages in python3, you should use pip3.
NOTE:- Its not necessary that pip will install in python 2.7, if python2 is absent then pip will do it in python3. The above statement was if you have both the version of python installed.
If you are a Linux user, then you can use the âwhichâ command to check which pip is installed. For example,
which pip
This command, if you enter into the Linux terminal, you will get to know about the version pip installed and the version of python. Otherwise, you may encounter an error if you have not installed pip.
To check for pip3:
which pip3
This command, if you enter into the Linux terminal, you will get to know about the version pip3 installed and the version of python. Otherwise, you may encounter an error if you have not installed pip3.
Conclusion
Python is one of the most intuitive programming languages for beginners, and itâs just generally a great programming language to have under your belt. However, to get the most out of it, youâll definitely want to get solid web hosting and understand the difference between pip vs pip3 using this easy guide.
If you have any doubts regarding todayâs article, feel free to comment down below. Till then, keep pythoning geeks!
The post PIP vs PIP3: What is the Difference? appeared first on Python Pool.
May 27, 2021
Python for Beginners
4 Ways to Read a Text File Line by Line in Python
Reading files is a necessary task in any programming language. Whether itâs a database file, image, or chat log, having the ability to read and write files greatly enhances what we can with Python.
Before we can create an automated audiobook reader or website, weâll need to master the basics. After all, no one ever ran before they crawled.
To complicate matters, Python offers several solutions for reading files. Weâre going to cover the most common procedures for reading files line by line in Python. Once youâve tackled the basics of reading files in Python, youâll be better prepared for the challenges that lie ahead.
Youâll be happy to learn that Python provides several functions for reading, writing, and creating files. These functions simplify file management by handling some of the work for us, behind the scenes.
We can use many of these Python functions to read a file line by line.
Read a File Line by Line with the readlines() Method
Our first approach to reading a file in Python will be the path of least resistance: the readlines() method. This method will open a file and split its contents into separate lines.
This method also returns a list of all the lines in the file. We can use readlines() to quickly read an entire file.
For example, letâs say we have a file containing basic information about employees at our company. We need to read this file and do something with the data.
employees.txt
Name: Marcus Gaye
Age: 25
Occupation: Web Developer
Name: Sally Rain
age: 31
Occupation: Senior Programmer
Example 1: Using readlines() to read a file
# open the data file
file = open("employees.txt")
# read the file as a list
data = file.readlines()
# close the file
file.close()
print(data)
Output
['Name: Marcus Gaye\n', 'Age: 25\n', 'Occupation: Web Developer\n', '\n', 'Name: Sally Rain\n', 'age: 31\n', 'Occupation: Senior Programmer\n']
readline() vs readlines()
Unlike its counterpart, the readline() method only returns a single line from a file. The realine() method will also add a trailing newline character to the end of the string.
With the readline() method, we also have the option of specifying a length for the returned line. If a size is not provided, the entire line will be read.
Consider the following text file:
wise_owl.txt
A wise old owl lived in an oak.
The more he saw the less he spoke.
The less he spoke the more he heard.
Why canât we all be like that wise old bird?
We can use readline() to get the first line of the text document. Unlike readlines(), only a single line will be printed when we use the readline() method to read the file.
Example 2: Read a single line with the readline() method
file = open("wise_owl.txt")
# get the first line of the file
line1 = file.readline()
print(line1)
file.close()
Output
A wise old owl lived in an oak.
The readline() method only retrieves a single line of text. Use readline() if you need to read all the lines at once.
file = open("wise_owl.txt")
# store all the lines in the file as a list
lines = file.readlines()
print(lines)
file.close()
Output
[âA wise old owl lived in an oak.\nâ, âThe more he saw the less he spoke.\nâ, âThe less he spoke the more he heard.\nâ, âWhy canât we all be like that wise old bird?\nâ]
Using a While Loop to Read a File
Itâs possible to read a file using loops as well. Using the same wise_owl.txt file that we made in the previous section, we can read every line in the file using a while loop.
Example 3: Reading files with a while loop and readline()
file = open("wise_owl.txt",'r')
while True:
next_line = file.readline()
if not next_line:
break;
print(next_line.strip())
file.close()
Output
A wise old owl lived in an oak.
The more he saw the less he spoke.
The less he spoke the more he heard.
Why canât we all be like that wise old bird?
Beware of infinite loops
A word of warning when working with while loops is in order. Be careful to add a termination case for the loop, otherwise youâll end up looping forever. Consider the following example:
while True:
print("Groundhog Day!")
Executing this code will cause Python to fall into an infinite loop, printing âGroundhog Dayâ until the end of time. When writing code like this, always provide a way to exit the loop.
If you find that youâve accidentally executed an infinite loop, you can escape it in the terminal by tapping Control+C on your keyboard.
Reading a file object in Python
Itâs also possible to read a file in Python using a for loop. For example, our client has given us a list of addresses of previous customers. We need to read the data using Python.
Hereâs the list of clients:
address_list.txt
Bobby Dylan
111 Longbranch Ave.
Houston, TX 77016
Sam Garfield
9805 Border Rd.
New Brunswick, NJ 08901
Penny Lane
408 2nd Lane
Lindenhurst, NY 11757
Marcus Gaye
622 Shub Farm St.
Rockledge, FL 32955
Prudence Brown
66 Ashley Ave.
Chaska, MN 55318
Whenever we open a file object, we can use a for loop to read its contents using the in keyword. With the in keyword, we can loop through the lines of the file.
Example 4: Using a for loop to read the lines in a file
# open the file
address_list = open("address_list.txt",'r')
for line in address_list:
print(line.strip())
address_list.close()
Unfortunately, this solution will not work for our client. Itâs very important that the data is in the form of a Python list. Weâll need to break the file into separate addresses and store them in a list before we can continue.
Example 5: Reading a file and splitting the content into a list
file = open("address_list.txt",'r')
address_list = []
i = 0
current_address = ""
for line in file:
# add a new address every three lines
if i > 2:
i = 0
address_list.append(current_address)
current_address = ""
else:
# add the line to the current address
current_address += line
i += 1
# use a for-in loop to print the list of addresses
for address in address_list:
print(address)
file.close()
Reading a file with the Context Manager
File management is a delicate procedure in any programming language. Files must be handled with care to prevent their corruption. When a file is opened, care must be taken to ensure the resource is later closed.
And there is a limit to how many files can be opened at once in Python. In order to avoid these problems, Python provides us with the Context Manager.
Introducing the with block
Whenever we open a file in Python, itâs important that we remember to close it. A method like readlines() will work okay for small files, but what if we have a more complex document? Using Python with statements will ensure that files are handled safely.
- The with statement is used for safely accessing resource files.
- Python creates a new context when it encounters the with block.
- Once the block executes, Python automatically closes the file resource.
- The context has the same scope as the with statement.
Letâs practice using the with statement by reading an email we have saved as a text file.
email.txt
Dear Valued Customer,
Thank you for reaching out to us about the issue with the product that you purchased. We are eager to address any concerns you may have about our products.
We want to make sure that you are completely satisfied with your purchase. To that end, we offer a 30-day, money-back guarantee on our entire inventory. Simply return the product and weâll happily refund the price of your purchase.
Thank you,
The ABC Company
code example
# open file
with open("email.txt",'r') as email:
# read the file with a for loop
for line in email:
# strip the newline character from the line
print(line.strip())
This time a for loop is used to read the lines of the file. When weâre using the Context Manager, the file is automatically closed when itâs handler goes out of scope. When the function is finished with the file, with statement ensures the resource is handled responsibly.
Summary
Weâve covered several ways of reading files line by line in Python. Weâve learned there is a big difference between the readline() and readlines() methods, and that we can use a for loop to read the contents of a file object.
We also learned how to use the with statement to open and read files. We saw how the context manager was created to make handling files safer and easier in Python.
Several examples were provided to illustrated the various forms of file handling available in Python. Take time to explore the examples and donât be afraid to experiment with the code if you donât understand something.
If youâd like to learn about programming with Python, follow the links below to view more great lessons from Python for Beginners.
Related Posts
- Using Python try except to handle errors
- How to use Python split string to improve your coding skills
- Using Python string concatenation for better readability
The post 4 Ways to Read a Text File Line by Line in Python appeared first on PythonForBeginners.com.
Mike Driscoll
PyInstaller â How to Turn Your Python Code into an Exe on Windows
You have just created an awesome new application. Maybe itâs a game or maybe itâs an image viewer. Whatever your application is, you want to share it with your friend or a family member. However, you know they wonât know how to install Python or any of the dependencies. What do you do? You need something that will transform your code into an executable!
Python has many different tools you can use to convert your Python code into a Windows executable. Here are a few different tools you can use:
- PyInstaller
- py2exe
- cx_freeze
- Nuitka
- Briefcase
These various tools can all be used to create executables for Windows. They work slightly differently, but the end result is that you will have an executable and perhaps some other files that you need to distribute too.
PyInstaller and Briefcase can be used to create Windows and MacOS executables. Nuitka is a little different in that it turns your Python code into C code before converting it into an executable. What this means is that the result ends up much smaller than PyInstallerâs executable.
However, for this article, you will focus on PyInstaller. It is one of the most popular packages for this purpose and has a lot of support. PyInstaller also has good documentation and there are many tutorials available for it.
In this article, you will learn about:
- Installing PyInstaller
- Creating an Executable for a Command-Line Application
- Creating an Executable for a GUI
Letâs transform some code into a Windows executable!
Installing PyInstaller
To get started, you will need to install PyInstaller. Fortunately, PyInstaller is a Python package that can be easily installed using pip:
python -m pip install pyinstaller
This command will install PyInstaller and any dependencies that it needs on your machine. You should now be ready to create an executable with PyInstaller!
Creating an Executable for a Command-Line Application
The next step is to pick some code that you want to turn into an executable. You can use the PySearch utility from chapter 32 of my book, Python 101: 2nd Edition, and turn it into a binary. Here is the code:
# pysearch.py
import argparse
import pathlib
def search_folder(path, extension, file_size=None):
"""
Search folder for files
"""
folder = pathlib.Path(path)
files = list(folder.rglob(f'*.{extension}'))
if not files:
print(f'No files found with {extension=}')
return
if file_size is not None:
files = [f for f in files
if f.stat().st_size > file_size]
print(f'{len(files)} *.{extension} files found:')
for file_path in files:
print(file_path)
def main():
parser = argparse.ArgumentParser(
'PySearch',
description='PySearch - The Python Powered File Searcher')
parser.add_argument('-p', '--path',
help='The path to search for files',
required=True,
dest='path')
parser.add_argument('-e', '--ext',
help='The extension to search for',
required=True,
dest='extension')
parser.add_argument('-s', '--size',
help='The file size to filter on in bytes',
type=int,
dest='size',
default=None)
args = parser.parse_args()
search_folder(args.path, args.extension, args.size)
if __name__ == '__main__':
main()
Next, open up a Command Prompt (cmd.exe) in Windows and navigate to the folder that has your pysearch.py file in it. To turn the Python code into a binary executable, you need to run the following command:
pyinstaller pysearch.py
If Python isnât on your Windows path, you may need to type out the full path to pyinstaller to get it to run. It will be located in a Scripts folder wherever your Python is installed on your system.
When you run that command, you will see some output that will look similar to the following:
6531 INFO: PyInstaller: 3.6 6576 INFO: Python: 3.8.2 6707 INFO: Platform: Windows-10-10.0.10586-SP0 6828 INFO: wrote C:\Users\mike\AppData\Local\Programs\Python\Python38-32\pysearch.spec 6880 INFO: UPX is not available. 7110 INFO: Extending PYTHONPATH with paths ['C:\\Users\\mike\\AppData\\Local\\Programs\\Python\\Python38-32', 'C:\\Users\\mike\\AppData\\Local\\Programs\\Python\\Python38-32'] 7120 INFO: checking Analysis 7124 INFO: Building Analysis because Analysis-00.toc is non existent 7128 INFO: Initializing module dependency graph... 7153 INFO: Caching module graph hooks... 7172 INFO: Analyzing base_library.zip ...
PyInstaller is very verbose and will print out a LOT of output. When it is finished, you will have a dist folder with a pysearch folder inside of it. Within the pysearch folder are many other files, including one called pysearch.exe. You can try navigating to the pysearch folder in your Command Prompt and then run pysearch.exe:
C:\Users\mike\AppData\Local\Programs\Python\Python38-32\dist\pysearch>pysearch.exe usage: PySearch [-h] -p PATH -e EXTENSION [-s SIZE] PySearch: error: the following arguments are required: -p/--path, -e/--ext
That looks like a pretty successful build! However, if you want to give the executable to your friends, you will have to give them the entire pysearch folder as all those other files in there are also required.
You can fix that issue by passing the --onefile flag, like this:
pyinstaller pysearch.py --onefile
The output from that command is similar to the first command. This time when you go into the dist folder though, you will find a single file in there called pysearch.exe instead of a folder full of files.
Creating an Executable for a GUI
Creating an executable for a GUI is slightly different than it is for a command-line application. The reason is that the GUI is the main interface and PyInstallerâs default is that the user will be using a Command Prompt or console window. If you run either of the PyInstaller commands that you learned about in the previous section, it will successfully create your executable. However, when you go to use your executable, you will see a Command Prompt appear in addition to your GUI.
You usually donât want that. To suppress the Command Prompt, you need to use the --noconsole flag.
To test out how this would work, grab the code for the image viewer that was created with wxPython from chapter 42 of Python 101: 2nd Edition. Here is the code again for your convenience:
# image_viewer.py
import wx
class ImagePanel(wx.Panel):
def __init__(self, parent, image_size):
super().__init__(parent)
self.max_size = 240
img = wx.Image(*image_size)
self.image_ctrl = wx.StaticBitmap(self,
bitmap=wx.Bitmap(img))
browse_btn = wx.Button(self, label='Browse')
browse_btn.Bind(wx.EVT_BUTTON, self.on_browse)
self.photo_txt = wx.TextCtrl(self, size=(200, -1))
main_sizer = wx.BoxSizer(wx.VERTICAL)
hsizer = wx.BoxSizer(wx.HORIZONTAL)
main_sizer.Add(self.image_ctrl, 0, wx.ALL, 5)
hsizer.Add(browse_btn, 0, wx.ALL, 5)
hsizer.Add(self.photo_txt, 0, wx.ALL, 5)
main_sizer.Add(hsizer, 0, wx.ALL, 5)
self.SetSizer(main_sizer)
main_sizer.Fit(parent)
self.Layout()
def on_browse(self, event):
"""
Browse for an image file
@param event: The event object
"""
wildcard = "JPEG files (*.jpg)|*.jpg"
with wx.FileDialog(None, "Choose a file",
wildcard=wildcard,
style=wx.ID_OPEN) as dialog:
if dialog.ShowModal() == wx.ID_OK:
self.photo_txt.SetValue(dialog.GetPath())
self.load_image()
def load_image(self):
"""
Load the image and display it to the user
"""
filepath = self.photo_txt.GetValue()
img = wx.Image(filepath, wx.BITMAP_TYPE_ANY)
# scale the image, preserving the aspect ratio
W = img.GetWidth()
H = img.GetHeight()
if W > H:
NewW = self.max_size
NewH = self.max_size * H / W
else:
NewH = self.max_size
NewW = self.max_size * W / H
img = img.Scale(NewW,NewH)
self.image_ctrl.SetBitmap(wx.Bitmap(img))
self.Refresh()
class MainFrame(wx.Frame):
def __init__(self):
super().__init__(None, title='Image Viewer')
panel = ImagePanel(self, image_size=(240,240))
self.Show()
if __name__ == '__main__':
app = wx.App(redirect=False)
frame = MainFrame()
app.MainLoop()
To turn this into an executable, you would run the following PyInstaller command:
pyinstaller.exe image_viewer.py --noconsole
Note that you are not using the --onefile flag here. Windows Defender will flag GUIs that are created with the --onefile as malware and remove it. You can get around that by not using the --onefile flag or by digitally signing the executable. Starting in Windows 10, all GUI applications need to be signed or they are considered malware.
Microsoft has a Sign Tool you can use, but you will need to purchase a digital certificate or create a self-signed certificate with Makecert, a .NET tool or something similar.
Wrapping Up
There are lots of different ways to create an executable with Python. In this article, you used PyInstaller. You learned about the following topics:
- Installing PyInstaller
- Creating an Executable for a Command-Line Application
- Creating an Executable for a GUI
PyInstaller has many other flags that you can use to modify its behavior when generating executables. If you run into issues with PyInstaller, there is a mailing list that you can turn to. Or you can search with Google and on StackOverflow. Most of the common issues that crop up are covered either in the PyInstaller documentation or are easily discoverable through searching online.
| Would you like to learn more about Python? |
Python 101 â 2nd Edition
Purchase now on Leanpub or Amazon
The post PyInstaller â How to Turn Your Python Code into an Exe on Windows appeared first on Mouse Vs Python.
death and gravity
When to use classes? When your functions take the same arguments
Are you having trouble figuring out when to use classes or how to organize them?
Have you repeatedly searched for "when to use classes in Python", read all the articles and watched all the talks, and still don't know whether you should be using classes in any given situation?
Have you read discussions about it that for all you know may be right, but they're so academic you can't parse the jargon?
Have you read articles that all treat the "obvious" cases, leaving you with no clear answer when you try to apply them to your own code?
My experience is that, unfortunately, the best way to learn this is to look at lots of examples.
Most guidelines tend to either be too vague if you don't already know enough about the subject, or too specific and saying things you already know.
This is one of those things that once you get it seems obvious and intuitive, but it's not, and is quite difficult to explain properly.
So, instead of prescribing a general approach, let's look at:
- one specific case where you may want to use classes
- examples from real-world code
- some considerations you should keep in mind
- The heuristic
- Example: HighlightedString
- Caveat: attribute changes are confusing
- Counter-example: single method
- Try it out
The heuristic #
If you have functions that take the same set of arguments, consider using a class.
That's it.
In its most basic form, a class is when you group data with functions that operate on that data; it doesn't have to represent a real ("business") object, it can be an abstract object that exists only to make things easier to use / understand.
Example: HighlightedString #
HighlightedString is a class I use in my feed reader library to wrap full-text search results, and highlight the matches when showing them on a web page.
At first, we need a function that takes a string and a list of slices1, and adds before/after markers to the parts inside the slices:
>>> value = 'water on mars'
>>> highlights = [slice(9, 13)]
>>> apply_highlights(value, highlights, '<b>', '</b>')
'water on <b>mars</b>'
While writing it, we pull part of the logic into a helper that splits the string such that highlights always have odd indices. We don't have to, but it's easier to reason about problems one at a time.
>>> list(split_highlights(value, highlights))
['water on ', 'mars', '']
To make things easier, we only allow non-overlapping slices with positive start/stop and no step. We pull this logic into another function that raises an exception for bad slices.
>>> validate_highlights(value, highlights) # no exception
>>> validate_highlights(value, [slice(6, 10), slice(9, 13)])
Traceback (most recent call last):
...
ValueError: highlights must not overlap: slice(6, 10, None), slice(9, 13, None)
Quiz: Which function should call validate_highlights()? Both? The user?
Instead of separate functions,
we can make apply() and split() methods on a class,
and do the validation in __init__:
>>> string = HighlightedString('water on mars', [slice(9, 13)])
>>> string.value
'water on mars'
>>> string.highlights
(slice(9, 13, None),)
>>>
>>> string.apply('<b>', '</b>')
'water on <b>mars</b>'
>>> list(string.split())
['water on ', 'mars', '']
>>>
>>> HighlightedString('water on mars', [slice(13, 9)])
Traceback (most recent call last):
...
ValueError: invalid highlight: start must be not be greater than stop: slice(13, 9, None)
Besides being shorter to use, this has a few additional benefits:
- it shows intent: this isn't just a string and some slices, it's a highlighted string
- it makes it easier to discover what actions are possible (help(), code completion)
- it makes code cleaner;
__init__validation ensures invalid objects cannot exist; thus, the methods don't have to validate anything themselves
Caveat: attribute changes are confusing #
Let's say we pass a highlighted string to a function that writes the results in a text file, and after that we do some other stuff with it.
What would you think if this happened?
>>> string.apply('<b>', '</b>')
'water on <b>mars</b>'
>>> render_results_page('output.txt', titles=[string])
>>> string.apply('<b>', '</b>')
'<b>water</b> on mars'
You may think it's quite unexpected; I know I would.
Either intentionally or by mistake,
render_results_page() seems to have changed our highlights,
when it was supposed to just render the results.
That's OK, mistakes happen. But how can we prevent it from happening in the future?
Solution: make the class immutable #
Well, in the real implementation, this mistake can't happen.
HighlightedString is a frozen dataclass,
which makes its attributes read-only;
also, highlights is always stored as a tuple,
which is immutable as well:
>>> string.highlights = [slice(0, 5)]
Traceback (most recent call last):
...
dataclasses.FrozenInstanceError: cannot assign to field 'highlights'
>>> string.highlights[:] = [slice(0, 5)]
Traceback (most recent call last):
...
TypeError: 'tuple' object does not support item assignment
You can find this pattern in werkzeug.datastructures, which contains HTTP-flavored subclasses of common Python objects. For example, Accept2 is an immutable list:
>>> accept = Accept([('image/png', 1)])
>>> accept[0]
('image/png', 1)
>>> accept.append(('image/gif', 1))
Traceback (most recent call last):
...
TypeError: 'Accept' objects are immutable
Counter-example: single method #
If you have a class with __init__ and one other method,
consider not using a class.
For example, you might implement a simple version of the command pattern like this:
class DoStuffCommand:
def __init__(self, path):
self.path = path
def execute(self):
... # do stuff with path
commands = [DoStuffCommand('some/path')]
for command in commands:
command.execute()
Python supports first-class functions, do this instead:
from functools import partial
def do_stuff(path):
... # do stuff with self.path
commands = [partial(do_stuff, 'some/path')]
for command in commands:
command()
If you you want more from your commands (e.g. undo support, or to validate the arguments before the command is executed), a class is still the way to go.
Try it out #
If you're doing something and you think you need a class, do it and see how it looks.
If you think it looks better, keep it; otherwise, revert the change. You can always switch in either direction later.
If you got it right the first time, great! If not, by having to fix it you'll learn something, and the next time you have a problem like this you'll know better.
Also, don't beat yourself up. Sure, there are nice libraries out there that use classes in just the right way, after spending lots of time to find the right abstraction. But abstraction is difficult and time consuming, and in everyday code good enough is just that â good enough â you don't need to go to the extreme.
That's it for now.
Found this useful? Consider sharing it wherever you share stuff, it really helps! :)
There are a few more examples I didn't have time to make just right, and will likely include in another article. If you want to be notified when it comes out, you can get updates via email or Atom feed.
A slice is an object Python uses internally for the extended indexing syntax;
thing[9:13]andthing[slice(9, 13)]are equivalent. [return]You may have used Accept yourself: the
request.accept_*attributes on Flask's request global are all Accept instances. [return]
Python Pool
Demystifying OpenCV keypoint in Python
OpenCV Library in python, which stands for Open Source Computer Vision, is a trendy library used for achieving artificial intelligence through python. Using the OpenCV library, we can process real-time images and videos for recognition and detection.
When humans see a particular human or animalâs image, we can recognize that image even when its orientation is changed or rotated. The reason behind it is that, as humans, we can easily identify the crucial keypoints in a given image. Thus, even if the image is rotated, we can still identify and relate it with the original picture because our brain can figure out the key points. Similarly, we need key points while processing images on the computer to identify the important points from an image. We achieve that by using OpenCV keypoint.
Importance of keypoints
While a computer is performing image processing, the computer should be able to identify similar characteristics in a given image irrespective of the transformations and rotations it goes through.
The computer should even be able to find similarities between different images belonging to a similar category. This is possible by observing the key points in a given image.
For example, for a human face, the key points are the two eye corners, two mouth corners, the chin, and the nose tip.
The main idea is that despite all the changes in an image, the computer should find the same key points in the new image.
The computer studies the pixel values surrounding a given key point and identifies the same when the images are modified.
OpenCV Keypoint functions
The OpenCV KeyPoint() function is used to store the keypoint found by several frameworks. The keypoint has characteristics such as orientation, scale, and 2D position.
The syntax of the OpenCV KeyPoint() is:
cv.KeyPoint(x, y, size[, angle[, response[, octave[, class_id]]]])
The Parameters are:
x: It is the position of the keypoint with respect to the x-axis
y: It is the position of the keypoint with respect to the y-axis
size: It is the diameter of the keypoint
angle: It is the orientation of the keypoint
response: Also known as the strength of the keypoint, it is the keypoint detector response on the keypoint
octave: It is the pyramid octave in which the keypoint has been detected
class_id: It is the object id
Apart from octave and class_id parameters, which have integer values, the other arguments have float values.
For drawing the KeyPoint, we use drawKeypoints() function. Its syntax is:
cv.drawKeypoints(image, keypoints, outImage[, color[, flags]])
Parameters:
image: It is the image from which keypoints have to be obtained
keypoints: These are the obtained keypoints from the source image
outImage: It is the output image
color: The color for the keypoints.
flags: Depending on the value set for flag, the keypoint is drawn accordingly.
Keypoint algorithms
SIFT, SURF, and ORB are algorithms that are used for finding keypoints while processing an image. There are several keypoints detected on an image, but we only use the important keypoints. We use a keypoint descriptor to identify which keypoint is important.
SIFT, SURF, and ORB detect keypoints as well as describe them. These techniques are invariant to transformations, scale changes, distortion, and rotation.
SIFT stands for Scale-Invariant Feature Transform. For SIFT, we use the âxâ coordinate, the âyâ coordinate, and the scale to detect the keypoints. We first resample the image by blurring the image and shrinking its size. Then, we perform scale invariance followed by rotation invariance. In rotation invariance, we calculate the orientation of the pixel around the keypoints. Then, we match the features of the prominent keypoints with other points in other images.
SURF stands for Speeded Up Robust Features. SURF, compared to SIFT, is more robust and faster while processing images. It uses a square-shaped filter because filtering an image with a square makes it a faster process. In addition, it uses a blob detector to find the keypoints. The blob detector is based on the hessian matrix. Overall, the surf algorithm has the same basic steps as the sift algorithm, but certain techniques make it faster.
ORB stands for Oriented FAST and Rotated BRIEF. First, it uses the FAST algorithm to find the keypoints. Then, we apply the Harris corner measure to find top N points. Finally, ORB uses a pyramid to produce multiscale features.
Example of OpenCV KeyPoint
KeyPoint determination is the fundamental concept behind all the applications of computer vision. Here, we will plot keypoints on a given image. We shall use the ORB algorithm for the same. First, we will import the cv2 library and import the cv2_imshow() function.
from google.colab.patches import cv2_imshow
import cv2
Now, we shall read the image using the imread() function. We have used a monkeyâs image. It is a colored image. So, we will be converting it into black and white by passing the flag value as zero.
Photo by Jamie Haughton on Unsplash
img = cv2.imread('monkey.jpg',0)
Now, we will use cv2.ORB_create() function. We shall pass 200 as the number of points we want.
orb = cv2.ORB_create(200)
Now we will use orb.detectAndCompute() to detect the keypoints and compute the descriptors. Finally, we will pass the image as an argument. It returns two values, the keypoints, and the descriptors.
keypoint, des = orb.detectAndCompute(img, None)
Using the drawKeypoints() function, we will plot all the keypoints. Then, we will pass the image, the keypoints, and the flag value as the arguments.
img_final = cv2.drawKeypoints(img, keypoint, None, flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
Finally, we will plot all the keypoints in out image by using cv2_imshow().
cv2_imshow(img_final)
The output image will be:
As we can see above, the important keypoints such as the mouth, two eyes, nose, right ear, and fingers have been detected by the computer.
FAQâs on OpenCV Keypoint
OpenCV keypoint is used in several computer vision applications such as human pose detection, human face detection, hand gesture detection,etc.
The detector is used to locate the keypoints from a given image. Whereas descriptors are used the describe the area around a potential keypoint.
That was everything about OpenCV Keypoints. If you have anything to share, let us know in the comments.
Till then, Keep Learning!
The post Demystifying OpenCV keypoint in Python appeared first on Python Pool.



