converting word to number in python

pip install word2number


from word2number import w2n
print(w2n.word_to_num('ten'))
10
print(w2n.word_to_num('one hundred twenty three'))
123
print(w2n.word_to_num('one thousand two hundred twenty three'))
1223

convert number to word in python

pip install n2w

import n2w

print(n2w.convert(12))

'twelve'

print(n2w.convert(123))

'one hundred twenty three'

underscores _ in Python

Understanding the underscore( _ ) of Python :

The underscore ( _ ) is special in Python.

There are 5 cases for using the underscore in Python.

• For storing the value of last expression in interpreter.

• For ignoring the specific values. (so-called “I don’t care”)

• To give special meanings and functions to name of variables or functions.

• To use as ‘Internationalization(i18n)’ or ‘Localization(l10n)’ functions.

• To separate the digits of number literal value.

When used in interpreter :

• The python interpreter stores the last expression value to the special variable called ‘_’.

Example:

>>> x=10

>>> x

10

>>> _

10

>>> _*2

20

>>> _

20

>>> _+3

23

>>> _

23

>>> y=_/3

>>> y

7.666666666666667


For Ignoring the values:

• The underscore is also used for ignoring the specific values. 

• If you don’t need the specific values or the values are not used, just assign the values to underscore.

Example:

Ignore a value when unpacking:

>>> x, _, y = (1, 2, 3)

>>> x

1

>>> y

3

>>> 

Ignore the multiple values:

• Note:which is available in only Python 3.x.

>>> x, *_, y = (1, 2, 3, 4, 5)

>>> x

1

>>> y

5

>>> 

Ignore the index:

for _ in range(10):

print("hai")

hai

hai

hai

hai

hai

hai

hai

hai

hai

hai

def do_something( ):

print("hello")

for _ in range(10):

do_something()

hello

hello

hello

hello

hello

hello

hello

hello

hello

hello

Ignore a value of specific location:

>>> x=(5,10,15,20,25)

>>> for _,val in x:

do_something()



Give special meanings to name of variables and functions:

• The underscore may be most used in ‘naming’.

• The PEP8 which is Python convention guideline introduces the following 4 naming cases.

single underscore( _ ):

• leading underscore( _ )

example:

_variablename or _functionname or _methodname or _classname

• This convention is used for declaring protected variables, functions, methods and classes in a module.

• tailing underscore( _ )

example:

variablename_ or functionname_ or methodname_ or classname_

• This convention could be used for avoiding conflict with Python keywords or built-ins.

Tkinter.Toplevel(master, class_='ClassName')
# Avoid conflict with 'class' keyword

list_ = List.objects.get(1)
# Avoid conflict with 'list' built-in type


double underscore( _ _ ):

• leading double underscore(_ _)

example:
 _ _varaiblename or _ _functionname or _ _methodname or _ _classname

• double underscore will mangle the attribute names of a class to avoide the conflicts of attribute names between classes.

• (so-called “mangling” that means that the compiler or interpreter modify the variables or function names with some rules, not use as it is). 

• The mangling rule of Python is adding the “_ClassName” to front of attribute names are declared with double underscore. 

• That is, if you write method named “__method” in a class, the name will be mangled in “_ClassName__method” form.

• leading and tailing double underscores(_ _)

example:
_ _variablename_ _ or _ _functionname_ _ or _ _classname_ _ or __methodname__

• This convention is used for special variables or methods (so-called “magic method”) such as   _ _init_ _  ,  _ _len__  ,          _ _new_ _ ,_ _del_ _,......etc.

• These methods provides special syntactic features or does special things.

example:
class A:
    def __init__(self, a):
          self.a = a
    def __custom__(self):
               pass


As Internationalization(i18n)/Localization(l10 n) functions :

• The built-in library gettext which is for i18n/l10n uses this convention, and Django which is Python web framework supports i18n/l10n also introduces and uses this convention.

# see official docs : https://docs.python.org/3/library/gettext.html import gettext
gettext.bindtextdomain('myapplication','/path/to/my/language /directory') gettext.textdomain('myapplication') _ = gettext.gettext
# ...
print(_('This is a translatable string.'))



To separate the digits of number literal value:

• This feature was added in Python 3.6. It is used for separating digits of numbers using underscore for readability.

>>> x=1_00

>>> x

100

>>> y="siva_krishna"

>>> y

'siva_krishna'

>>> z=0b_101

>>> bin(z)

'0b101'
>>> 

manual debugging in python

python debugging


import  pdb

for i in range(0,10):

            x=i*3

           print(i,x)

           pdb.set_trace()

Note:

n  means next

c  means continue

x  print current x value

i  print current i value


output:

0 0

> d:\siva krishna\python_practice\python tricks\debuggin.py(3)<module>()

-> for i in range(0,10):

(Pdb) i

0

(Pdb) x

0
(Pdb) c

1 3

> d:\siva krishna\python_practice\python tricks\debuggin.py(3)<module>()

-> for i in range(0,10):

(Pdb) i

1

(Pdb) x

3

(Pdb) c

2 6

> d:\siva krishna\python_practice\python tricks\debuggin.py(3)<module>()

-> for i in range(0,10):

(Pdb) i

2

(Pdb) x

6

(Pdb) n

> d:\siva krishna\python_practice\python tricks\debuggin.py(4)<module>()

-> x=i*3

(Pdb) c

3 9

> d:\siva krishna\python_practice\python tricks\debuggin.py(3)<module>()

-> for i in range(0,10):

(Pdb) i

3

(Pdb) x

9


(Pdb) 

clear the python command window screen

how to clear the python command window screen


C:\Users\siva>python

Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (
Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>>> print("hai siva")
hai siva
>>> print("welcome to my blog")
welcome to my blog
>>> x=10
>>> y=20
>>> z=x+y
>>> z
30
>>> import  os
>>> def  cls( ):
...     os.system("CLS")
...
>>> cls( )

>>>

Sending email with python

• sending email with python, first we need to install module called smtplib

• in latest versions of python smtplib module is builtin

import smtplib

host="smtp.gmail.com"

port=587

username="abc@gmail.com"

password="*********"

from_email=username

to_list=["xyz@gmail.com","abc@gmail.com"]

email_conn=smtplib.SMTP(host,port)

email_conn.ehlo()

email_conn.starttls()

email_conn.login(username,password)

email_conn.sendmail(from_email,to_list,

                    "hello,good morning")

print("mail successfully send")

email_conn.quit()

Pass by value and Pass by reference in python

Pass by value (call by value) and Pass by reference (call by reference) :

Pass by value (call by value):

•  Original value is not modified in call by value.

•  In call by value, value being passed to the function is locally stored by the function parameter in stack memory location. 

•  If you change the value of function parameter, it is changed for the current function only.

Example:

a=10

def f1(x):

    print(x)

    x=20

    print(x)

f1(a)

print(a)

Output:

10

20

10

pass by reference (call by reference):

• In call by reference, original value is modified because we pass reference (address).

• Here, address of the value is passed in the function, so actual and formal arguments shares the same address space. 

• value changed inside the function, is reflected inside as well as outside the function.

Example:

b=[1,2,3]

def f2(y):

    print(y)

    y[0]=10

    y[1]=20

    print(y)

f2(b)

print(b)

Output:

[1, 2, 3]

[10, 20, 3]

[10, 20, 3]

The concept of if _ _name_ _ = = '_ _main_ _ condition

The concept of      if _ _name_ _ = =  '_ _main_ _ ' :

Example:  addition.py

def  add(x,y):

     z=x+y

     print(z)


add(4,5)

Output:

9

Example: adding.py

from addition import add


add(2,3)

Output:

9

5

Note:

• here executing both main module and sub module add function.

• in these cases we don't want execute imported module add function(sub module add function). 

• i want execute only main module add function.

in this situation we are using   if  _ _name_ _ = = '_ _main_ _' :

Example:

def add(x,y):

    z=x+y

    print(z)

if __name__= = '__main__':

    add(4,5)

Example:

from addition import add


add(2,3)

Output:

5

Access modifiers in Python

Access modifiers in Python :

In C++ and Java, things are pretty straight-forward. There are 3 magical and easy to remember access modifiers, that will do the job (public, protected and private). 

But there’s no such a thing in Python. That might be a little confusing at first, but it’s possible too.

Public:

All member variables and methods are public by default in Python. So when you want to make your member public, you just do nothing.

Protected:

Protected member is (in C++ and Java) accessible only from within the class and it’s sub classes.

How to accomplish this in Python? 

The answer is _ by convention. By prefixing the name of your member with a single underscore, you’re telling others “don’t touch this, unless you’re a subclass”.

• This changes virtually nothing, you’ll still be able to access the variable from outside the class, only if you see something like this.

• you explain politely to the person responsible for this, that the variable is protected and he should not access it or even worse, change it from outside the class.

Private:

By declaring your data member private you mean, that nobody should be able to access it from outside the class, i.e. strong you can’t touch this policy. 

Python supports a technique called name mangling. This feature turns every member name prefixed with at least two underscores and suffixed with at most one underscore into 

_<className>_ _<memberName>  ------> Name Mangling

Example:

class test:

    def  __init__(self): #constructor

        self.x=10  #public

        self._y=20  #protected

        self.__z=30  #private

    def  m1(self): #method

        print(self.x)

 

        print(self._y) 

        print(self.__z) 

    

t1=test( )

t1.m1( )

print(t1.x)

print(t1._y)

print(t1._test__z)

print(dir(t1))

Output:

10

20

30

10

20

30

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_test__z', '_y', 'm1', 'x']

How to check whether number is prime number or not

Prime Number:
Any number divisible by 1 and itself that number is called prime number.

Example:

num = int(input("enter number: "))
if num > 1:
   for i in range(2,num):
       if (num % i) == 0:
           print(num,"is not a prime number")
           print(i,"times",num//i,"is",num)
           break
   else:
       print(num,"is a prime number")
       

else:
   print(num,"is not a prime number")


Output:

enter number: 5
5 is a prime number

Output:

enter number: 12
12 is not a prime number
2 times 6 is 12

Output:

enter number: 15
15 is not a prime number
3 times 5 is 15

how to calculate Age

write a program to calculate age in python:

import  datetime
year=datetime.datetime.now().year
year_of_birth=int(input("enter your birth year: "))
print("you are %i years old!"%(year-year_of_birth))

Output:

enter your birth year: 1991
you are 26 years old!

check whether given number is Armstrong or not

Armstrong number: the sum of cubes of each digit in a number is equal to original number.

Example:
                     153=1³ + 5³ + 3^{3
                                   1 +  125 + 27}

Example:

num=int(input("enter a number: "))
sum = 0
temp = num
while temp > 0:
   digit = temp % 10
   sum += digit ** 3
   temp //= 10
if num == sum:
   print(num,"is an Armstrong number")
else:
   print(num,"is not an Armstrong number")

Output:

enter a number: 153

153 is an Armstrong number

Example 2: 
find the Armstrong numbers between given range

lower = 100
upper = 2000
for num in range(lower, upper + 1):  
   order = len(str(num))   
   sum = 0   
   temp = num
   while temp > 0:
       digit = temp % 10
       sum += digit ** order
       temp //= 10
   if num == sum:

       print(num)

Output:

153

370

371

407

1634

how to check python version at run time

Example:

import sys

print(sys.version)

Output:

3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)]

how to check given string is palindrome or not

Check whether given string is palindrome or not:

Example:

def palindrome(string):
    if string==string[ : : -1]:
        return True
    else:
        return False
print(palindrome("madam"))
print(palindrome("siva"))

print(palindrome("anna"))

Output:

True
False

True

for loop with else in python

for loop with else:

Example1:

for i in range(5):
    if i<5:
        print("hai")
else:

    print("hello")

Output:

hai
hai
hai
hai
hai

hello

Example2:

for i in range(5):
    if i==2:
        break
    print("hai")
else:

    print("hello")

Output:

hai

hai

Example3:

for i in range(5):

    if i==2:

        continue

    print("hai",i)

else:

    print("hello")

Output:

hai 0

hai 1

hai 3

hai 4

hello

switch case implementation in python

Switch Case:

def switch(operator,x,y):
    if operator=='add':
        z=x+y
        print("the addition of",x,"and",y,"is: ",z)
    elif operator=='sub':
        z=x-y
        print("the subtraction of",x,"and",y,"is: ",z)
    elif operator=='mul':
        z=x*y
        print("the multiplication of",x,"and",y,"is: ",z)
    elif operator=='div':
        z=x/y
        print("the division of",x,"and",y,"is: ",z)
    else:
        print("operation failed")
switch('add',4,5)
switch('sub',4,5)
switch('mul',4,5)
switch('div',4,5)
switch('mod',4,5)

Output:

        
the addition of 4 and 5 is:  9
the subtraction of 4 and 5 is:  -1
the multiplication of 4 and 5 is:  20
the division of 4 and 5 is:  0.8

operation failed

Scope of the variables in python

Scope of the variables in python:

 we are following the rule: LEGB

L means Local

E means Enclosing

G means Global

B means Builtin

Example:

x="global x"
y="global y"
z="global z"
def f1(x):
    y="local y"
    z="local z"
    x="local x"
    print(y)
    print(z)
    print(x)

f1("enclosing z")

Output:

loacl y
local z

local x

Example:

x="global x"
y="global y"
z="global z"
def f1(z):
    y="local y"
    print(y)
    print(z)
    print(x)

f1("enclosing z")

Output:

loacl y
enclosing z

global x

Example:

x="global x"
y="global y"
z="global z"
def f1(z,x):
    y="loacl y"
    print(y)
    z=20
    print(z)
    print(x)

f1("enclosing z","enclosing x")

Output:

loacl y
20

enclosing x

Sentence capitalizer in python

Example:

def f1 (string1: str):
    sentences = string1.split(". ")
    sentences2 = [sentence[0].capitalize() + sentence[1:] for sentence in sentences]
    string2 = '. '.join(sentences2)
    return string2
print (f1("hello. my name is Joe. what is your name?"))

Output:

Hello. My name is Joe. What is your name?

Example:

import re
def sentence_case(text):    
     
    sentences = re.findall('[^.!?]+[.!?](?:\s|\Z)', text)

    sentences = [x[0].upper() + x[1:] for x in sentences]
   
    return ''.join(sentences)

print(sentence_case("hEllo. my name is Joe. what is your name?"))

Output:

HEllo. My name is Joe. What is your name?

finding maximum and minimum numbers by using lambda function

Finding maximum number:

my_max=lambda a,b,c:a if a>b and a>c else(b if b>c else c)

print(my_max(4,7,6))

Output:

7

Finding minimum number:

my_min=lambda a,b,c:a if a<b and a<c else(b if b<c else c)

print(my_min(4,7,6))



Output:

4

Fibonacci series

write Fibonacci series up to n

def fib(n):
    a, b = 0, 1
    while b < n:
        print(b,)
        a, b = b, a+b
fib(100)

Output:

1

1

2

3

5

8

13

21

34

55

89

swapping of two numbers in python

swapping of two numbers by using temporary variable

a=int(input("enter a value: "))

b=int(input("enter b value: "))

print("before swapping: ","a=",a,"b=",b)

temp=a

a=b

b=temp

print("after swapping: ","a=",a,"b=",b)

Output:

enter a value: 4

enter b value: 5

before swapping:  a= 4 b= 5


after swapping:  a= 5 b= 4



swapping of two numbers 

a=int(input("enter a value: "))

b=int(input("enter b value: "))

print("before swapping: ","a=",a,"b=",b)

a,b=b,a

print("after swapping: ","a=",a,"b=",b)

Output:

enter a value: 4

enter b value: 5

before swapping:  a= 4 b= 5


after swapping:  a= 5 b= 4

how to check memory size of object in python

check memory size of objects:

import sys
sys.getsizeof(object)

note: 
memory represents in bytes

python 2.7.13

>>> import sys
>>> x=2
>>> type(x)
<type 'int'>
>>> sys.getsizeof(x)
12
>>> y=2.3
>>> sys.getsizeof(y)
16
>>> z=3+4j
>>> sys.getsizeof(z)
24
>>> a=True
>>> sys.getsizeof(a)
12
>>> b="siva"
>>> sys.getsizeof(b)
25
>>> c=[1,2,3]
>>> sys.getsizeof(c)
48
>>> c=[10,2.3,"siva"]
>>> sys.getsizeof(c)
48
>>> d=(1,2,3)
>>> sys.getsizeof(d)
40
>>> e={1,2,3}
>>> sys.getsizeof(e)
116
>>> f={'maths':75,'science':80}
>>> sys.getsizeof(f)
140
>>> 

python 3.6.2

>>> import sys
>>> x=2
>>> sys.getsizeof(x)
28
>>> y=2.3
>>> sys.getsizeof(y)
24
>>> z=3+4j
>>> sys.getsizeof(z)
32
>>> a=True
>>> sys.getsizeof(a)
28
>>> b="siva"
>>> sys.getsizeof(b)
53
>>> c=[1,2,3]
>>> sys.getsizeof(c)
88
>>> d=(1,2,3)
>>> sys.getsizeof(d)
72
>>> e={1,2,3}
>>> sys.getsizeof(e)
224
>>> f={'maths':75,'science':80}
>>> sys.getsizeof(f)
240

>>> 

merging or adding of two dictionaries

• merging or adding of two dictionaries this feature available from python 3.5 + versions on words.

Example:

x={'a':1,'b':2}

y={'c':3,'d':4}

z={'a':5,'b':6}

p={'a':5,'b':7}

xy={**x,**y}

print(xy)

xz={**x,**z}

print(xz)

zp={**z,**p}

print(zp)

Output:

{'a': 1, 'b': 2, 'c': 3, 'd': 4}

{'a': 5, 'b': 6}

{'a': 5, 'b': 7}

args and kwargs in python

Normal Arguments and arbitrary arguments and keyword arguments in python:

• Normal argument type depends on value/data

• Arbitrary argument by default type is tuple, Represents *

• keyword argument  by default type is dictionary,Represents **

Example:

def f1(req,*args,**kwargs):
    if req:
        print(req,type(req))
    if args:
        print(args,type(args))
    if kwargs:
        print(kwargs,type(kwargs))

f1(10)

f1(10,2.3,5,"siva")

f1(10,2.3,5,"siva",key1=20,key2="siva",key3=True)

Output:

10  <class 'int'>

10  <class 'int'>
(2.3, 5, 'siva')  <class 'tuple'>

10  <class 'int'>
(2.3, 5, 'siva')  <class 'tuple'>

{'key1': 20, 'key2': 'siva', 'key3': True}  <class 'dict'>

constructors in python

_ _new_ _ 

• __new__ is for creating objects.

• Use __new__ when you need to control the creation of a new instance.

• __new__ is the first step of instance creation. It's called first, and is responsible for returning a new instance of your class.

• __new__ is static, class method .

• __new__ takes cls as parameter.

• __new__ is good for immutable object as they cannot be changed once they are assigned. So we can return new instance which has new state.

class A:

    

    def __new__(cls):

        print "A.__new__ is called"  # -> this is never called

A( )

_ _init_ _

• __init__ is for initializing objects.

•  Use __init__ when you need to control initialization of a new instance.

• In contrast, __init__ doesn't return anything,it's only responsible for initializing the instance after it's been created.

• __init__ is instance method. 

• __init__ takes self as parameter.

• __init__ called on it automatically.

class A:

    def __init__(self):
        print "A.__init__ called"


A( )



class A:

    def __init__(self):
        return 29


A( )


class A(object):  # -> don't forget the object specified as base

    def __new__(cls):
        print "A.__new__ called"
        return super(A, cls).__new__(cls)

    def __init__(self):
        print "A.__init__ called"


A( )

Example:

class ExampleClass(object):

     def __new__(cls, value):

         print"Creating new instance..."

         #Call the superclass constructor to create the instance.

         instance = super(ExampleClass, cls).__new__(cls)

         return instance

     def __init__(self, value):

        print"Initialising instance..."

        self.x= value

exampleInstance = ExampleClass(42)

print(exampleInstance.x)

why method takes self parameter in python

when objects are instantiated,the object itself is passed in to the self parameter of the method.

class test:

     def m1(self):

           print("hai")

t1=test()

t1.m1()

note:

• t1 passed to the self parameter of the m1( ) method

methods in python

Methods:

• It is possible to define two kinds of methods with-in a class that can be called without an instance.

Class Method:

• A class method takes cls as first parameter.
• A class method can access or modify class state.
• We use @classmethod decorator in python to create a class method.
• We generally use class method to create factory methods.

Syntax:

class C(object):

    @classmethod

    def fun(cls, arg1, arg2, ...):

       ........................

       ........................

       ........................

Static Method:

• A static method needs no specific parameters.
• A static method can’t access or modify it.
• We use @staticmethod decorator to create a static method in python.
• We generally use static methods to create utility functions.

Syntax:

class C(object):

    @staticmethod

    def fun( arg1, arg2, ...):

       ....................

       ....................

       ....................

Example:

class Test(object):

    x=20

    def m1(self):

        print"hai,i am in instance method"

        print(Test.x)

 

    @classmethod

    def m2(cls):

        print "hai,i am in class method"

        print(Test.x)

 

    @staticmethod

    def m3():

        print "hai,i am in static method"

        print(Test.x)

 

Test.m2()

Test.m3()

Test.m1()

web scraping

• When performing data science tasks, it’s common to want to use data found on the internet. 

• You’ll usually be able to access this data in csv format, or via an Application Programming Interface(API). 

• However, there are some times when the data you want can only be accessed as part of a web page. 

• In cases like this, you’ll want to use a technique called web scraping. 

Web-Scraping:

• to get the data from the web page into a format you can work with in your analysis.

• We are going to use Python as our scraping language, together with a simple and powerful library, BeautifulSoup.

pip install BeautifulSoup4 or beautifulsoup4

Before we start jumping into the code, let’s understand the basics of HTML and some rules of scraping.

<!DOCTYPE html>  ----> type deceleration
<html>  
    <head>
    </head>
    <body>
        <h1> First Scraping </h1>
        <p> Hello World </p>
    <body>

</html>


This is the basic syntax of an HTML webpage. Every <tag> serves a block inside the webpage:

1. <!DOCTYPE html>: HTML documents must start with a type declaration.

2. The HTML document is contained between <html> and </html>.

3. The meta and script declaration of the HTML document is between <head> and </head>.

4. The visible part of the HTML document is between <body> and </body> tags.

5. Title headings are defined with the <h1> through <h6> tags.

6. Paragraphs are defined with the <p> tag.

• Other useful tags include <a> for hyperlinks, <table> for tables, <tr> for table rows, and <td> for table columns.

• Also, HTML tags sometimes come with id or class attributes. 

• The id attribute specifies a unique id for an HTML tag and the value must be unique within the HTML document. 

• The class attribute is used to define equal styles for HTML tags with the same class. We can make use of these ids and classes to help us locate the data we want.

Scraping Rules:

• You should check a website’s Terms and Conditions before you scrape it. 

• Be careful to read the statements about legal use of data. 

• Usually, the data you scrape should not be used for commercial purposes.

• Do not request data from the website too aggressively with your program (also known as spamming), as this may break the website. 

• Make sure your program behaves in a reasonable manner (i.e. acts like a human). 

• One request for one webpage per second is good practice.

• The layout of a website may change from time to time, so make sure to revisit the site and rewrite your code as needed.

Examle:1

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

print(soup)

Example:2

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

print(source)

Example:3

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

print(soup.title)

Example:4

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

print(soup.title.string)

Example:5

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

print(soup.title.text)

Example:6

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( ) 

soup=bs.BeautifulSoup(source,'lxml')

print(soup.p)

Example:7

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

print(soup.find_all('p'))

Example:8

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

for paragraph in soup.find_all('p'):

    print(paragraph)

Example:9

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

for paragraph in soup.find_all('p'):

    print(paragraph.string)

Example:10

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

for paragraph in soup.find_all('p'):

    print(paragraph.text)

Example:11

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

print(soup.get_text( ))

Example:12

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

for url in soup.find_all('a'):

    print(url.text)

Example:13

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

for url in soup.find_all('a'):

    print(url.get('href'))

Example:14

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

nav=soup.nav

print(nav)

Example:15

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

nav=soup.nav

for url in nav.find_all('a'):

    print(url.get('href'))

Example:16

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

body=soup.body

for paragraph in body.find_all('p'):

    print(paragraph.text)

Example:17

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

for div in soup.find_all('div'):

    print(div.text)

Example:18

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

for div in soup.find_all('div',class_='body'):

    print(div.text)

Example:19

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

table=soup.table

print(table)

Example:20

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

table=soup.find('table')

print(table)

Example:21

import bs4 as bs

import urllib

source=urllib.urlopen('https://pythonprogramming.net/parsememcparseface/').read( )

soup=bs.BeautifulSoup(source,'lxml')

table=soup.find('table')

table_rows=table.find_all('tr')

for tr in table_rows:

    td=tr.find_all('td')

    row=[i.text for i in td]

    print(row)

Example:22

import bs4 as bs

import urllib

import pandas as pd

dfs=pd.read_html('https://pythonprogramming.net/parsememcparseface/')

for df in dfs:

    print(df)

Example:23

import bs4 as bs

import urllib

import pandas as pd

dfs=pd.read_html('https://pythonprogramming.net/parsememcparseface/',header=0)

for df in dfs:

    print(df)

Example:24

import bs4 as bs

import urllib

import pandas as pd

source=urllib.urlopen('https://pythonprogramming.net/sitemap.xml').read()

soup=bs.BeautifulSoup(source,'xml')

print(soup)

Example:25

import bs4 as bs

import urllib

import pandas as pd

source=urllib.urlopen('https://pythonprogramming.net/sitemap.xml').read()

soup=bs.BeautifulSoup(source,'xml')

for url in soup.find_all('loc'):

    print(url.text)

Advanced Scraping Techniques:

• BeautifulSoup is simple and great for small-scale web scraping, But if you are interested in scraping data at a larger scale, you should consider using these other alternatives: Scrapy

• Try to integrate your code with some public APIs. The efficiency of data retrieval is much higher than scraping webpages. 

• For example, take a look at Facebook Graph API, which can help you get hidden data which is not shown on Facebook webpages.

• Consider using a database backend like MySQL to store your data when it gets too large.