opencv-python-tutroals


Чтобы посмотреть этот PDF файл с форматированием и разметкой, скачайте его и откройте на своем компьютере.
OpenCV-PythonTutorials
Documentation
Release1
AlexanderMordvintsev&AbidK
Sep24,2017
Contents
1OpenCV-PythonTutorials
3
1.1IntroductiontoOpenCV
.........................................
6
1.2GuiFeaturesinOpenCV
.........................................
19
1.3CoreOperations
.............................................
34
1.4ImageProcessinginOpenCV
......................................
46
i
ii
OpenCV-PythonTutorialsDocumentation,Release1
Contents:
Contents
1
OpenCV-PythonTutorialsDocumentation,Release1
2
Contents
CHAPTER
1
OpenCV-PythonTutorials

IntroductiontoOpenCV

GuiFeaturesinOpenCV
Hereyouwilllearnhowtodisplayandsaveimagesandvideos,control
mouseeventsandcreatetrackbar.

CoreOperations
Inthissectionyouwilllearnbasicoperationsonimagelikepixelediting,

ImageProcessinginOpenCV
3
OpenCV-PythonTutorialsDocumentation,Release1
Inthissectionyouwilllearndifferentimageprocessingfunctionsinside
OpenCV.


VideoAnalysis
Inthissectionyouwilllearndifferenttechniquestoworkwithvideoslike

CameraCalibrationand3DReconstruction

MachineLearning
Inthissectionyouwilllearndifferentimageprocessingfunctionsinside
OpenCV.

ComputationalPhotography
Inthissectionyouwilllearndifferentcomputationalphotographytech-

4Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1

OpenCV-PythonBindings
Inthissection,wewillseehowOpenCV-Pythonbindingsaregenerated
5
OpenCV-PythonTutorialsDocumentation,Release1
IntroductiontoOpenCV

IntroductiontoOpenCV-PythonTutorials

InstallOpenCV-PythoninWindows

InstallOpenCV-PythoninFedora
6Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
IntroductiontoOpenCV-PythonTutorials
OpenCV
OpenCVwasstartedatIntelin1999by
GaryBradsky
andtherstreleasecameoutin2000.
VadimPisarevsky
joinedGaryBradskytomanageIntel'sRussiansoftwareOpenCVteam.In2005,OpenCVwasusedonStanley,the
vehiclewhowon2005DARPAGrandChallenge.LateritsactivedevelopmentcontinuedunderthesupportofWillow
Garage,withGaryBradskyandVadimPisarevskyleadingtheproject.Rightnow,OpenCVsupportsalotofalgorithms
relatedtoComputerVisionandMachineLearninganditisexpandingday-by-day.
1.1.IntroductiontoOpenCV
7
OpenCV-PythonTutorialsDocumentation,Release1
Andthatwillbeagoodtaskforfresherswhobegintocontributetoopensourceprojects.JustforktheOpenCV
ingithub,makenecessarycorrectionsandsendapullrequesttoOpenCV.OpenCVdeveloperswillcheckyourpull
request,giveyouimportantfeedbackandonceitpassestheapprovalofthereviewer,itwillbemergedtoOpenCV.
8Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
8.
Copy
cv2.pyd
to
C:/Python27/lib/site-packeges
.
9.
OpenPythonIDLEandtypefollowingcodesinPythonterminal.
���
import
cv2
���
print
cv2
.
__version__
Iftheresultsareprintedoutwithoutanyerrors,congratulations!!!YouhaveinstalledOpenCV-Pythonsuccessfully.
BuildingOpenCVfromsource
1.
DownloadandinstallVisualStudioandCMake.
1.1.
VisualStudio2012
1.2.
CMake
2.
DownloadandinstallnecessaryPythonpackagestotheirdefaultlocations
2.1.
Python2.7.x
2.2.
Numpy
2.3.
Matplotlib
(
Matplotlibisoptional,butrecommendedsinceweuseitalotinourtutorials.
)
Note:
Inthiscase,weareusing32-bitbinariesofPythonpackages.ButifyouwanttouseOpenCVforx64,64-bit
binariesofPythonpackagesaretobeinstalled.Problemisthat,thereisnoofcial64-bitbinariesofNumpy.You
havetobuilditonyourown.Forthat,youhavetousethesamecompilerusedtobuildPython.WhenyoustartPython
Note:
3.
MakesurePythonandNumpyareworkingne.
4.
DownloadOpenCVsource.Itcanbefrom
Sourceforge
(forofcialreleaseversion)orfrom
Github
(forlatest
source).
5.
Extractittoafolder,
opencv
andcreateanewfolder
build
init.
6.
OpenCMake-gui(
Start�AllPrograms�CMake-gui
)
7.
Filltheeldsasfollows(seetheimagebelow):
7.1.Clickon
BrowseSource...
andlocatethe
opencv
folder.
7.2.Clickon
BrowseBuild...
andlocatethe
build
folderwecreated.
7.3.Clickon
Congure
.
1.1.IntroductiontoOpenCV
9
OpenCV-PythonTutorialsDocumentation,Release1
7.4.Itwillopenanewwindowtoselectthecompiler.Chooseappropriatecompiler(here,Visual
Studio11)andclick
Finish
.
7.5.Waituntilanalysisisnished.
8.
Youwillseealltheeldsaremarkedinred.Clickonthe
WITH
eldtoexpandit.Itdecideswhatextrafeatures
youneed.Somarkappropriateelds.Seethebelowimage:
10Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
9.
Nowclickon
BUILD
1.1.IntroductiontoOpenCV11
OpenCV-PythonTutorialsDocumentation,Release1
10.
12Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
11.
Nowclickon
ENABLE
eldtoexpandit.Makesure
ENABLE_SOLUTION_FOLDERS
isunchecked(So-
lutionfoldersarenotsupportedbyVisualStudioExpressedition).Seetheimagebelow:
1.1.IntroductiontoOpenCV13
OpenCV-PythonTutorialsDocumentation,Release1
12.
Alsomakesurethatinthe
PYTHON
eld,everythingislled.(IgnorePYTHON_DEBUG_LIBRARY).See
imagebelow:
13.
Finallyclickthe
Generate
button.
14.
Nowgotoour
opencv/build
folder.Thereyouwillnd
OpenCV.sln
le.OpenitwithVisualStudio.
15.
Checkbuildmodeas
Release
insteadof
Debug
.
16.
Inthesolutionexplorer,right-clickonthe
Solution
(or
ALL_BUILD
)andbuildit.Itwilltakesometimeto
nish.
17.
Again,right-clickon
INSTALL
andbuildit.NowOpenCV-Pythonwillbeinstalled.
14Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
18.
OpenPythonIDLEandenter
importcv2
.Ifnoerror,itisinstalledcorrectly.
Note:
AdditionalResources
Exercises
1.
$yuminstallnumpyopencv
*
OpenPythonIDLE(orIPython)andtypefollowingcodesinPythonterminal.
���
import
cv2
���
print
cv2
.
__version__
Iftheresultsareprintedoutwithoutanyerrors,congratulations!!!YouhaveinstalledOpenCV-Pythonsuccessfully.
Itisquiteeasy.Butthereisaproblemwiththis.YumrepositoriesmaynotcontainthelatestversionofOpenCV
always.Forexample,atthetimeofwritingthistutorial,yumrepositorycontains2.4.5whilelatestOpenCVversionis
1.1.IntroductiontoOpenCV15
OpenCV-PythonTutorialsDocumentation,Release1
InstallingOpenCVfromsource
Compilingfromsourcemayseemalittlecomplicatedatrst,butonceyousucceededinit,thereisnothingcompli-
cated.
Firstwewillinstallsomedependencies.Somearecompulsory,someareoptional.Optionaldependencies,youcan
leaveifyoudon'twant.
CompulsoryDependencies
Weneed
CMake
toconguretheinstallation,
GCC
forcompilation,
Python-devel
and
Numpy
forcreatingPython
yuminstallcmake
yuminstallpython-develnumpy
yuminstallgccgcc-c++
Nextweneed
GTK
supportforGUIfeatures,Camerasupport(libdc1394,libv4l),MediaSupport(ffmpeg,gstreamer)
yuminstallgtk2-devel
yuminstalllibdc1394-devel
yuminstalllibv4l-devel
yuminstallffmpeg-devel
yuminstallgstreamer-plugins-base-devel
OptionalDependencies
AbovedependenciesaresufcienttoinstallOpenCVinyourfedoramachine.Butdependinguponyourrequirements,
youmayneedsomeextradependencies.Alistofsuchoptionaldependenciesaregivenbelow.Youcaneitherleaveit
orinstallit,yourcall:)
yuminstalllibpng-devel
yuminstalllibjpeg-turbo-devel
yuminstalljasper-devel
yuminstallopenexr-devel
yuminstalllibtiff-devel
yuminstalllibwebp-devel
SeveralOpenCVfunctionsareparallelizedwith
Intel'sThreadingBuildingBlocks
(TBB).Butifyouwanttoen-
yuminstalltbb-devel
OpenCVusesanotherlibrary
Eigen
foroptimizedmathematicaloperations.SoifyouhaveEigeninstalledinyoursys-
yuminstalleigen3-devel
16Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Ifyouwanttobuild
documentation
(
yuminstallpython-sphinx
yuminstalltexlive
DownloadingOpenCV
NextwehavetodownloadOpenCV.YoucandownloadthelatestreleaseofOpenCVfrom
sourceforgesite
.Then
extractthefolder.
OryoucandownloadlatestsourcefromOpenCV'sgithubrepo.(IfyouwanttocontributetoOpenCV,choosethis.It
alwayskeepsyourOpenCVup-to-date).Forthat,youneedtoinstall
Git
rst.
yuminstallgit
gitclonehttps://github.com/Itseez/opencv.git
Itwillcreateafolder
OpenCV
inhomedirectory(orthedirectoryyouspecify).Thecloningmaytakesometime
mkdirbuild
cd
build
ConguringandInstalling
cmake-D
CMAKE_BUILD_TYPE
=
RELEASE-D
CMAKE_INSTALL_PREFIX
=
/usr/local..
Itspeciesthatbuildtypeis“ReleaseMode”andinstallationpathis
/usr/local
.Observethe
-D
beforeeach
optionand
..
attheend.Inshort,thisistheformat:
cmake
[
-Dlag;flag
]
[
-Dlag;flag
]
..
Youcanspecifyasmanyagsyouwant,buteachagshouldbeprecededby
-D
.
Sointhistutorial,weareinstallingOpenCVwithTBBandEigensupport.Wealsobuildthedocumentation,butwe
excludePerformancetestsandbuildingsamples.WealsodisableGPUrelatedmodules(sinceweuseOpenCV-Python,
wedon'tneedGPUrelatedmodules.Itsavesussometime).
cmake-D
WITH_TBB
=
ON-D
WITH_EIGEN
=
ON..
1.1.IntroductiontoOpenCV17
OpenCV-PythonTutorialsDocumentation,Release1

Enabledocumentationanddisabletestsandsamples
cmake-D
BUILD_DOCS
=
ON-D
BUILD_TESTS
=
OFF-D
BUILD_PERF_TESTS
=
OFF-D
,
!
BUILD_EXAMPLES
=
OFF..

DisableallGPUrelatedmodules.
cmake-D
WITH_OPENCL
=
OFF-D
WITH_CUDA
=
OFF-D
BUILD_opencv_gpu
=
OFF-D
,
!
BUILD_opencv_gpuarithm
=
OFF-D
BUILD_opencv_gpubgsegm
=
OFF-D
BUILD_
,
!
opencv_gpucodec
=
OFF-D
BUILD_opencv_gpufeatures2d
=
OFF-D
BUILD_opencv_
,
!
gpufilters
=
OFF-D
BUILD_opencv_gpuimgproc
=
OFF-D
BUILD_opencv_
,
!
gpulegacy
=
OFF-D
BUILD_opencv_gpuoptflow
=
OFF-D
BUILD_opencv_
,
!
gpustereo
=
OFF-D
BUILD_opencv_gpuwarping
=
OFF..

cmake-D
CMAKE_BUILD_TYPE
=
RELEASE-D
CMAKE_INSTALL_PREFIX
=
/usr/local..
--GUI:
--GTK+
2
.x:YES
(
ver
2
.24.19
)
--GThread:YES
(
ver
2
.36.3
)
--VideoI/O:
--DC1394
2
.x:YES
(
ver
2
.2.0
)
--FFMPEG:YES
--codec:YES
(
ver
54
.92.100
)
--format:YES
(
ver
54
.63.104
)
--util:YES
(
ver
52
.18.100
)
--swscale:YES
(
ver
2
.2.100
)
--gentoo-style:YES
--GStreamer:
--base:YES
(
ver
0
.10.36
)
--video:YES
(
ver
0
.10.36
)
--app:YES
(
ver
0
.10.36
)
--riff:YES
(
ver
0
.10.36
)
--pbutils:YES
(
ver
0
.10.36
)
--V4L/V4L2:Usinglibv4l
(
ver
1
.0.0
)
--Otherthird-partylibraries:
--UseEigen:YES
(
ver
3
.1.4
)
--UseTBB:YES
(
ver
4
.0interface
6004
)
--Python:
18Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
--PdfLaTeXcompiler:/usr/bin/pdflatex
--
--Testsandsamples:
--Tests:NO
--Performancetests:NO
--C/C++Examples:NO
make
su
makeinstall
Installationisover.Alllesareinstalledin
/usr/local/
folder.Buttouseit,yourPythonshouldbeabletond
OpenCVmodule.Youhavetwooptionsforthat.
1.
MovethemoduletoanyfolderinPythonPath
:Pythonpathcanbefoundoutbyentering
import
sys;printsys.path
inPythonterminal.Itwillprintoutmanylocations.Move
/usr/local/lib/
python2.7/site-packages/cv2.so
toanyofthisfolder.Forexample,
sumv/usr/local/lib/python2.7/site-packages/cv2.so/usr/lib/python2.7/
,
!
site-packages
ButyouwillhavetodothiseverytimeyouinstallOpenCV.
2.
Add``/usr/local/lib/python2.7/site-packages``tothePYTHON_PATH
:Itistobedoneonlyonce.Justopen
~/.bashrc
andaddfollowinglinetoit,thenlogoutandcomeback.
export
PYTHONPATH
=
$PYTHONPATH
:/usr/local/lib/python2.7/site-packages
ThusOpenCVinstallationisnished.Openaterminalandtry
importcv2
.
Tobuildthedocumentation,justenterfollowingcommands:
makedocs
makehtml_docs
Thenopen
opencv/build/doc/_html/index.html
andbookmarkitinthebrowser.
AdditionalResources
Exercises
1.
CompileOpenCVfromsourceinyourFedoramachine.
GuiFeaturesinOpenCV

1.2.GuiFeaturesinOpenCV19
OpenCV-PythonTutorialsDocumentation,Release1
Learntoloadanimage,displayitandsaveitback

Learntoplayvideos,capturevideosfromCameraandwriteitasavideo

DrawingFunctionsinOpenCV

MouseasaPaint-Brush
Drawstuffswithyourmouse

20Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Note:
Insteadofthesethreeags,youcansimplypassintegers1,0or-1respectively.
Seethecodebelow:
import
numpy
as
np
import
cv2
#Loadancolorimageingrayscale
img
=
cv2
.
imread(

messi5.jpg

,
0
)
Warning:
Eveniftheimagepathiswrong,itwon'tthrowanyerror,but
printimg
willgiveyou
None
Displayanimage
Usethefunction
cv2.imshow()
todisplayanimageinawindow.Thewindowautomaticallytstotheimagesize.
Firstargumentisawindownamewhichisastring.secondargumentisourimage.Youcancreateasmanywindows
asyouwish,butwithdifferentwindownames.
cv2
.
imshow(

image

,img)
cv2
.
waitKey(
0
)
cv2
.
destroyAllWindows()
Ascreenshotofthewindowwilllooklikethis(inFedora-Gnomemachine):
1.2.GuiFeaturesinOpenCV21
OpenCV-PythonTutorialsDocumentation,Release1
cv2.waitKey()
isakeyboardbindingfunction.Itsargumentisthetimeinmilliseconds.Thefunctionwaitsfor
speciedmillisecondsforanykeyboardevent.Ifyoupressanykeyinthattime,theprogramcontinues.If
0
ispassed,
Note:
Thereisaspecialcasewhereyoucanalreadycreateawindowandloadimagetoitlater.Inthatcase,youcan
Seethecodebelow:
cv2
.
namedWindow(

image

,cv2
.
WINDOW_NORMAL)
cv2
.
imshow(

image

,img)
cv2
.
waitKey(
0
)
cv2
.
destroyAllWindows()
Writeanimage
Usethefunction
cv2.imwrite()
tosaveanimage.
Firstargumentisthelename,secondargumentistheimageyouwanttosave.
cv2
.
imwrite(

messigray.png

,img)
ThiswillsavetheimageinPNGformatintheworkingdirectory.
22Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Sumitup
Belowprogramloadsanimageingrayscale,displaysit,savetheimageifyoupress`s'andexit,orsimplyexitwithout
savingifyoupress
ESC
key.
import
numpy
as
np
import
cv2
img
=
cv2
.
imread(

messi5.jpg

,
0
)
cv2
.
imshow(

image

,img)
k
=
cv2
.
waitKey(
0
)
if
k
==
27
:
#waitforESCkeytoexit
cv2
.
destroyAllWindows()
elif
k
==
ord
(

s

):
#waitforskeytosaveandexit
cv2
.
imwrite(

messigray.png

,img)
cv2
.
destroyAllWindows()
Warning:
Ifyouareusinga64-bitmachine,youwillhavetomodify
k=cv2.waitKey(0)
lineasfollows:
k=cv2.waitKey(0)&0xFF
UsingMatplotlib
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

messi5.jpg

,
0
)
plt
.
imshow(img,cmap
=

gray

,interpolation
=

bicubic

)
plt
.
xticks([]),plt
.
yticks([])
#tohidetickvaluesonXandYaxis
plt
.
show()
Ascreen-shotofthewindowwilllooklikethis:
1.2.GuiFeaturesinOpenCV23
OpenCV-PythonTutorialsDocumentation,Release1
Seealso:
Warning:
ColorimageloadedbyOpenCVisinBGRmode.ButMatplotlibdisplaysinRGBmode.Socolor
imageswillnotbedisplayedcorrectlyinMatplotlibifimageisreadwithOpenCV.Pleaseseetheexercisesfor
AdditionalResources
1.
MatplotlibPlottingStylesandFeatures
Exercises
1.
ThereissomeproblemwhenyoutrytoloadcolorimageinOpenCVanddisplayitinMatplotlib.Read
this
discussion
andunderstandit.
24Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Goal

Learntoreadvideo,displayvideoandsavevideo.

LearntocapturefromCameraanddisplayit.

Youwilllearnthesefunctions:
cv2.VideoCapture()
,
cv2.VideoWriter()
CaptureVideofromCamera
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(
0
)
while
(
True
):
#Captureframe-by-frame
cap.read()
Note:
1.2.GuiFeaturesinOpenCV25
OpenCV-PythonTutorialsDocumentation,Release1
PlayingVideofromle
ItissameascapturingfromCamera,justchangecameraindexwithvideolename.Alsowhiledisplayingtheframe,
useappropriatetimefor
cv2.waitKey()
.Ifitistooless,videowillbeveryfastandifitistoohigh,videowillbe
slow(Well,thatishowyoucandisplayvideosinslowmotion).25millisecondswillbeOKinnormalcases.
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(

vtest.avi

)
while
(cap
.
isOpened()):
Note:
SavingaVideo
Sowecaptureavideo,processitframe-by-frameandwewanttosavethatvideo.Forimages,itisverysimple,just
use
cv2.imwrite()
.Herealittlemoreworkisrequired.
Thistimewecreatea
VideoWriter
object.Weshouldspecifytheoutputlename(eg:output.avi).Thenweshould
specifythe
FourCC
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(
0
)
#DefinethecodecandcreateVideoWriterobject
26Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
fourcc
=
cv2
.
VideoWriter_fourcc(
*

XVID

)
out
=
cv2
.
VideoWriter(

output.avi

,fourcc,
20.0
,(
640
,
480
))
while
(cap
.
isOpened()):
AdditionalResources
Exercises
DrawingFunctionsinOpenCV
Goal

1.2.GuiFeaturesinOpenCV27
OpenCV-PythonTutorialsDocumentation,Release1
import
numpy
as
np
import
cv2
#Createablackimage
img
=
np
.
zeros((
512
,
512
,
3
),np
.
uint8)
#Drawadiagonalbluelinewiththicknessof5px
img
=
cv2
.
line(img,(
0
,
0
),(
511
,
511
),(
255
,
0
,
0
),
5
)
DrawingRectangle
Todrawarectangle,youneedtop-leftcornerandbottom-rightcornerofrectangle.Thistimewewilldrawagreen
rectangleatthetop-rightcornerofimage.
img
=
cv2
.
rectangle(img,(
384
,
0
),(
510
,
128
),(
0
,
255
,
0
),
3
)
DrawingCircle
Todrawacircle,youneeditscentercoordinatesandradius.Wewilldrawacircleinsidetherectangledrawnabove.
img
=
cv2
.
circle(img,(
447
,
63
),
63
,(
0
,
0
,
255
),
-
1
)
DrawingEllipse
Todrawtheellipse,weneedtopassseveralarguments.Oneargumentisthecenterlocation(x,y).Nextargumentis
axeslengths(majoraxislength,minoraxislength).
angle
istheangleofrotationofellipseinanti-clockwisedirec-
tion.
startAngle
and
endAngle
denotesthestartingandendingofellipsearcmeasuredinclockwisedirection
img
=
cv2
.
ellipse(img,(
256
,
256
),(
100
,
50
),
0
,
0
,
180
,
255
,
-
1
)
DrawingPolygon
Todrawapolygon,rstyouneedcoordinatesofvertices.Makethosepointsintoanarrayofshape
ROWSx1x2
where
ROWSarenumberofverticesanditshouldbeoftype
int32
.Herewedrawasmallpolygonofwithfourverticesin
yellowcolor.
pts
=
np
.
array([[
10
,
5
],[
20
,
30
],[
70
,
20
],[
50
,
10
]],np
.
int32)
pts
=
pts
.
reshape((
-
1
,
1
,
2
))
img
=
cv2
.
polylines(img,[pts],
True
,(
0
,
255
,
255
))
Note:
Ifthirdargumentis
False
Note:
cv2.polylines()
canbeusedtodrawmultiplelines.Justcreatealistofallthelinesyouwanttodraw
28Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
linesthancalling
cv2.line()
foreachline.
AddingTexttoImages:
Toputtextsinimages,youneedspecifyfollowingthings.

Textdatathatyouwanttowrite

Positioncoordinatesofwhereyouwantputit(i.e.bottom-leftcornerwheredatastarts).

Fonttype(Check
cv2.putText()
docsforsupportedfonts)

FontScale(speciesthesizeoffont)

font
=
cv2
.
FONT_HERSHEY_SIMPLEX
cv2
.
putText(img,

OpenCV

,(
10
,
500
),font,
4
,(
255
,
255
,
255
),
2
,cv2
.
LINE_AA)
Result
Soitistimetoseethenalresultofourdrawing.Asyoustudiedinpreviousarticles,displaytheimagetoseeit.
1.2.GuiFeaturesinOpenCV29
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
30Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
SimpleDemo
Here,wecreateasimpleapplicationwhichdrawsacircleonanimagewhereverwedouble-clickonit.
Firstwecreateamousecallbackfunctionwhichisexecutedwhenamouseeventtakeplace.Mouseeventcanbe
���
import
cv2
���
events
=
[i
for
i
in
dir
(cv2)
if

EVENT

in
i]
���
print
events
Creatingmousecallbackfunctionhasaspecicformatwhichissameeverywhere.Itdiffersonlyinwhatthefunction
does.Soourmousecallbackfunctiondoesonething,itdrawsacirclewherewedouble-click.Soseethecodebelow.
Codeisself-explanatoryfromcomments:
import
cv2
import
numpy
as
np
#mousecallbackfunction
def
draw_circle
(event,x,y,flags,param):
if
event
==
cv2
.
EVENT_LBUTTONDBLCLK:
cv2
.
circle(img,(x,y),
100
,(
255
,
0
,
0
),
-
1
)
#Createablackimage,awindowandbindthefunctiontowindow
img
=
np
.
zeros((
512
,
512
,
3
),np
.
uint8)
cv2
.
namedWindow(

image

)
cv2
.
MoreAdvancedDemo
import
cv2
import
numpy
as
np
drawing
=
False
#trueifmouseispressed
mode
=
True
#ifTrue,drawrectangle.Pressmtotoggletocurve
ix,iy
=
-
1
,
-
1
#mousecallbackfunction
def
draw_circle
(event,x,y,flags,param):
global
ix,iy,drawing,mode
if
event
==
cv2
.
EVENT_LBUTTONDOWN:
drawing
=
True
1.2.GuiFeaturesinOpenCV31
OpenCV-PythonTutorialsDocumentation,Release1
ix,iy
=
x,y
elif
event
==
cv2
.
EVENT_MOUSEMOVE:
if
drawing
==
True
:
if
mode
==
True
:
cv2
.
rectangle(img,(ix,iy),(x,y),(
0
,
255
,
0
),
-
1
)
else
:
cv2
.
circle(img,(x,y),
5
,(
0
,
0
,
255
),
-
1
)
elif
event
==
cv2
.
EVENT_LBUTTONUP:
drawing
=
False
if
mode
==
True
:
cv2
.
rectangle(img,(ix,iy),(x,y),(
0
,
255
,
0
),
-
1
)
else
:
cv2
.
circle(img,(x,y),
5
,(
0
,
0
,
255
),
-
1
)
img
=
np
.
zeros((
512
,
512
,
3
),np
.
uint8)
cv2
.
namedWindow(

image

)
cv2
.
AdditionalResources
Exercises
1.
Inourlastexample,wedrewlledrectangle.Youmodifythecodetodrawanunlledrectangle.
32Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
import
cv2
import
numpy
as
np
def
nothing
(x):
pass
#Createablackimage,awindow
img
=
np
.
zeros((
300
,
512
,
3
),np
.
uint8)
cv2
.
namedWindow(

image

)
#createtrackbarsforcolorchange
cv2
.
Thescreenshotoftheapplicationlookslikebelow:
1.2.GuiFeaturesinOpenCV33
OpenCV-PythonTutorialsDocumentation,Release1
Exercises
1.
CreateaPaintapplicationwithadjustablecolorsandbrushradiususingtrackbars.Fordrawing,referprevious
tutorialonmousehandling.
CoreOperations

BasicOperationsonImages
Learntoreadandeditpixelvalues,workingwithimageROIandotherbasic
operations.

34Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1

PerformanceMeasurementandImprovementTechniques

MathematicalToolsinOpenCV
LearnsomeofthemathematicaltoolsprovidedbyOpenCVlikePCA,SVD
1.3.CoreOperations
35
OpenCV-PythonTutorialsDocumentation,Release1
BasicOperationsonImages
Goal
Learnto:

Accesspixelvaluesandmodifythem

Accessimageproperties

���
import
cv2
���
import
numpy
as
np
���
img
=
cv2
.
imread(

messi5.jpg

)
���
px
=
img[
100
,
100
]
���
print
px
[157166200]
#accessingonlybluepixel
���
blue
=
img[
100
,
100
,
0
]
���
print
blue
157
Youcanmodifythepixelvaluesthesameway.
���
img[
100
,
100
]
=
[
255
,
255
,
255
]
���
print
img[
100
,
100
]
[255255255]
Warning:
Numpyisaoptimizedlibraryforfastarraycalculations.Sosimplyaccessingeachandeverypixel
valuesandmodifyingitwillbeveryslowanditisdiscouraged.
Note:
36Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
#accessingREDvalue
��

img
.
item(
10
,
10
,
2
)
59
#modifyingREDvalue
��

img
.
AccessingImageProperties
���
print
img
.
shape
(342,548,3)
Note:
Totalnumberofpixelsisaccessedby
img.size
:
���
print
img
.
size
562248
Imagedatatypeisobtainedby
img.dtype
:
���
print
img
.
dtype
uint8
Note:
img.dtype
isveryimportantwhiledebuggingbecausealargenumberoferrorsinOpenCV-Pythoncodeis
causedbyinvaliddatatype.
ImageROI
image:
���
ball
=
img[
280
:
340
,
330
:
390
]
���
img[
273
:
333
,
100
:
160
]
=
ball
Checktheresultsbelow:
1.3.CoreOperations
37
OpenCV-PythonTutorialsDocumentation,Release1
SplittingandMergingImageChannels
TheB,G,Rchannelsofanimagecanbesplitintotheirindividualplaneswhenneeded.Then,theindividualchannels
���
b,g,r
=
cv2
.
split(img)
���
img
=
cv2
.
merge((b,g,r))
Or
���
b
=
img[:,:,
0
]
Suppose,youwanttomakealltheredpixelstozero,youneednotsplitlikethisandputitequaltozero.Youcan
simplyuseNumpyindexingwhichisfaster.
���
img[:,:,
2
]
=
0
Warning:
cv2.split()
isacostlyoperation(intermsoftime),soonlyuseitifnecessary.Numpyindexing
ismuchmoreefcientandshouldbeusedifpossible.
MakingBordersforImages(Padding)
38Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1

borderType-Flagdeningwhatkindofbordertobeadded.Itcanbefollowingtypes:

cv2.BORDER_CONSTANT
-Addsaconstantcoloredborder.Thevalueshouldbegivenasnext
argument.

cv2.BORDER_REFLECT
-Borderwillbemirrorreectionoftheborderelements,likethis:
fed-
cba|abcdefgh|hgfedcb

cv2.BORDER_REFLECT_101
or
cv2.BORDER_DEFAULT
-Sameasabove,butwithaslight
change,likethis:
gfedcb|abcdefgh|gfedcba

cv2.BORDER_REPLICATE
-Lastelementisreplicatedthroughout,likethis:
aaaaaa|abcdefgh|hhhhhhh

cv2.BORDER_WRAP
-Can'texplain,itwilllooklikethis:
cdefgh|abcdefgh|abcdefg

value
-Colorofborderifbordertypeis
cv2.BORDER_CONSTANT
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
BLUE
=
[
255
,
0
,
0
]
img1
=
cv2
.
imread(

opencv_logo.png

)
replicate
=
cv2
.
copyMakeBorder(img1,
10
,
10
,
10
,
10
,cv2
.
BORDER_REPLICATE)
reflect
=
cv2
.
copyMakeBorder(img1,
10
,
10
,
10
,
10
,cv2
.
BORDER_REFLECT)
reflect101
=
cv2
.
copyMakeBorder(img1,
10
,
10
,
10
,
10
,cv2
.
BORDER_REFLECT_101)
wrap
=
cv2
.
copyMakeBorder(img1,
10
,
10
,
10
,
10
,cv2
.
BORDER_WRAP)
constant
=
cv2
.
copyMakeBorder(img1,
10
,
10
,
10
,
10
,cv2
.
BORDER_CONSTANT,value
=
BLUE)
plt
.
subplot(
231
),plt
.
imshow(img1,

gray

),plt
.
title(

ORIGINAL

)
plt
.
subplot(
232
),plt
.
imshow(replicate,

gray

),plt
.
title(

REPLICATE

)
plt
.
subplot(
233
),plt
.
imshow(reflect,

gray

),plt
.
title(

REFLECT

)
plt
.
subplot(
234
),plt
.
imshow(reflect101,

gray

),plt
.
title(

REFLECT_101

)
plt
.
subplot(
235
),plt
.
imshow(wrap,

gray

),plt
.
title(

WRAP

)
plt
.
subplot(
236
),plt
.
imshow(constant,

gray

),plt
.
title(

CONSTANT

)
plt
.
show()
Seetheresultbelow.(Imageisdisplayedwithmatplotlib.SoREDandBLUEplaneswillbeinterchanged):
1.3.CoreOperations
39
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
Note:
40Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Forexample,considerbelowsample:
���
x
=
np
.
uint8([
250
])
���
y
=
np
.
uint8([
10
])
���
print
cv2
.
add(x,y)
#250+10=260�=255
[[255]]
���
print
x
+
y
#250+10=260%256=4
[4]
img1
=
cv2
.
imread(

ml.png

)
img2
=
cv2
.
imread(

opencv_logo.jpg

)
dst
=
cv2
.
addWeighted(img1,
0.7
,img2,
0.3
,
0
)
cv2
.
imshow(

dst

,dst)
cv2
.
waitKey(
0
)
cv2
.
destroyAllWindows()
Checktheresultbelow:
1.3.CoreOperations
41
OpenCV-PythonTutorialsDocumentation,Release1
BitwiseOperations
ThisincludesbitwiseAND,OR,NOTandXORoperations.Theywillbehighlyusefulwhileextractinganypartof
#Loadtwoimages
img1
=
cv2
.
imread(

messi5.jpg

)
img2
=
cv2
.
imread(

opencv_logo.png

)
#Iwanttoputlogoontop-leftcorner,SoIcreateaROI
rows,cols,channels
=
img2
.
shape
roi
=
img1[
0
:rows,
0
:cols]
#Nowcreateamaskoflogoandcreateitsinversemaskalso
img2gray
=
cv2
.
cvtColor(img2,cv2
.
COLOR_BGR2GRAY)
Seetheresultbelow.Leftimageshowsthemaskwecreated.Rightimageshowsthenalresult.Formoreunderstand-
ing,displayalltheintermediateimagesintheabovecode,especially
img1_bg
and
img2_fg
.
42Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
1.
e1
=
cv2
.
Wewilldemonstratewithfollowingexample.Followingexampleapplymedianlteringwithakernelofoddsize
rangingfrom5to49.(Don'tworryaboutwhatwilltheresultlooklike,thatisnotourgoal):
img1
=
cv2
.
imread(

messi5.jpg

)
e1
=
cv2
.
1.3.CoreOperations
43
OpenCV-PythonTutorialsDocumentation,Release1
Note:
Youcandothesamewith
time
module.Insteadof
DefaultOptimizationinOpenCV
#checkifoptimizationisenabled
In[
5
]:cv2
.
useOptimized()
Out[
5
]:
True
In[
6
]:
%
timeitres
=
cv2
.
medianBlur(img,
49
)
10
loops,bestof
3
:
34.9
msperloop
#Disableit
In[
7
]:cv2
.
See,optimizedmedianlteringis~2xfasterthanunoptimizedversion.Ifyoucheckitssource,youcanseemedian
lteringisSIMDoptimized.Soyoucanusethistoenableoptimizationatthetopofyourcode(rememberitisenabled
bydefault).
MeasuringPerformanceinIPython
In[
10
]:x
=
5
In[
11
]:
%
timeity
=
x
*
*
2
10000000
loops,bestof
3
:
73
nsperloop
In[
12
]:
%
timeity
=
x
*
x
10000000
loops,bestof
3
:
58.3
nsperloop
In[
15
]:z
=
np
.
uint8([
5
])
In[
17
]:
%
timeity
=
z
*
z
1000000
loops,bestof
3
:
1.25
usperloop
44Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
In[
19
]:
%
timeity
=
np
.
square(z)
1000000
loops,bestof
3
:
1.16
usperloop
Youcanseethat,
x=5;y=x
*
x
isfastestanditisaround20xfastercomparedtoNumpy.Ifyouconsiderthe
arraycreationalso,itmayreachupto100xfaster.Cool,right?
(Numpydevsareworkingonthisissue)
Note:
PythonscalaroperationsarefasterthanNumpyscalaroperations.Soforoperationsincludingoneortwo
Wewilltryonemoreexample.Thistime,wewillcomparetheperformanceof
cv2.countNonZero()
and
np.count_nonzero()
forsameimage.
In[
35
]:
%
timeitz
=
cv2
.
countNonZero(img)
100000
loops,bestof
3
:
15.8
usperloop
In[
36
]:
%
timeitz
=
np
.
count_nonzero(img)
1000
loops,bestof
3
:
370
usperloop
See,OpenCVfunctionisnearly25xfasterthanNumpyfunction.
Note:
Normally,OpenCVfunctionsarefasterthanNumpyfunctions.Soforsameoperation,OpenCVfunctionsare
preferred.But,therecanbeexceptions,especiallywhenNumpyworkswithviewsinsteadofcopies.
MoreIPythonmagiccommands
Thereareseveralothermagiccommandstomeasuretheperformance,proling,lineproling,memorymeasurement
1.3.CoreOperations
45
OpenCV-PythonTutorialsDocumentation,Release1
3.
TimingandProlinginIPython
Exercises
MathematicalToolsinOpenCV
ImageProcessinginOpenCV

ChangingColorspaces


ImageThresholding
Learntoconvertimagestobinaryimagesusingglobalthresholding,Adap-

SmoothingImages

MorphologicalTransformations
46Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
LearnaboutmorphologicaltransformationslikeErosion,Dilation,Open-

ImageGradients


ImagePyramids
Learnaboutimagepyramidsandhowtousethemforimageblending

ContoursinOpenCV
AllaboutContoursinOpenCV

HistogramsinOpenCV
1.4.ImageProcessinginOpenCV47
OpenCV-PythonTutorialsDocumentation,Release1
AllabouthistogramsinOpenCV

ImageTransformsinOpenCV

TemplateMatching
LearntosearchforanobjectinanimageusingTemplateMatching

HoughLineTransform

HoughCircleTransform

ImageSegmentationwithWatershedAlgorithm
48Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Learntosegmentimageswithwatershedsegmentation

InteractiveForegroundExtractionusingGrabCutAlgorithm
LearntoextractforegroundwithGrabCutalgorithm
1.4.ImageProcessinginOpenCV49
OpenCV-PythonTutorialsDocumentation,Release1
ChangingColorspaces
Goal

Inthistutorial,youwilllearnhowtoconvertimagesfromonecolor-spacetoanother,likeBGR
$
Gray,BGR
$
���
import
cv2
���
flags
=
[i
for
i
in
dir
(cv2)
if
i
.
startswith(

COLOR_

)]
���
print
flags
Note:
ForHSV,Huerangeis[0,179],Saturationrangeis[0,255]andValuerangeis[0,255].Differentsoftwaresuse
differentscales.SoifyouarecomparingOpenCVvalueswiththem,youneedtonormalizetheseranges.
ObjectTracking
NowweknowhowtoconvertBGRimagetoHSV,wecanusethistoextractacoloredobject.InHSV,itismoreeasier
torepresentacolorthanRGBcolor-space.Inourapplication,wewilltrytoextractabluecoloredobject.Sohereis
import
cv2
import
numpy
as
np
cap
=
cv2
.
VideoCapture(
0
)
while
(
1
):
#Takeeachframe
_,frame
=
cap
.
read()
#ConvertBGRtoHSV
50Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
hsv
=
cv2
.
cvtColor(frame,cv2
.
COLOR_BGR2HSV)
#definerangeofbluecolorinHSV
lower_blue
=
np
.
array([
110
,
50
,
50
])
upper_blue
=
np
.
array([
130
,
255
,
255
])
Belowimageshowstrackingoftheblueobject:
Note:
Therearesomenoisesintheimage.Wewillseehowtoremovetheminlaterchapters.
Note:
HowtondHSVvaluestotrack?
Thisisacommonquestionfoundinstackoverow.com.Itisverysimpleandyoucanusethesamefunction,
cv2.cvtColor()
.Insteadofpassinganimage,youjustpasstheBGRvaluesyouwant.Forexample,tondthe
1.4.ImageProcessinginOpenCV51
OpenCV-PythonTutorialsDocumentation,Release1
HSVvalueofGreen,tryfollowingcommandsinPythonterminal:
���
green
=
np
.
uint8([[[
0
,
255
,
0
]]])
���
hsv_green
=
cv2
.
cvtColor(green,cv2
.
COLOR_BGR2HSV)
���
print
hsv_green
[[[60255255]]]
Nowyoutake[H-10,100,100]and[H+10,255,255]aslowerboundandupperboundrespectively.Apartfromthis
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

gradient.png

,
0
)
52Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Note:
Toplotmultipleimages,wehaveused
plt.subplot()
Resultisgivenbelow:
AdaptiveThresholding
Intheprevioussection,weusedaglobalvalueasthresholdvalue.Butitmaynotbegoodinalltheconditionswhere
imagehasdifferentlightingconditionsindifferentareas.Inthatcase,wegoforadaptivethresholding.Inthis,the
1.4.ImageProcessinginOpenCV53
OpenCV-PythonTutorialsDocumentation,Release1

cv2.ADAPTIVE_THRESH_MEAN_C:thresholdvalueisthemeanofneighbourhoodarea.

cv2.ADAPTIVE_THRESH_GAUSSIAN_C:thresholdvalueistheweightedsumofneighbourhoodval-
ueswhereweightsareagaussianwindow.
BlockSize
-Itdecidesthesizeofneighbourhoodarea.
C
-Itisjustaconstantwhichissubtractedfromthemeanorweightedmeancalculated.
Belowpieceofcodecomparesglobalthresholdingandadaptivethresholdingforanimagewithvaryingillumination:
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

dave.jpg

,
0
)
img
=
cv2
.
medianBlur(img,
5
)
Result:
54Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Otsu'sBinarization
1.4.ImageProcessinginOpenCV55
OpenCV-PythonTutorialsDocumentation,Release1
Checkoutbelowexample.Inputimageisanoisyimage.Inrstcase,Iappliedglobalthresholdingforavalueof
127.Insecondcase,IappliedOtsu'sthresholdingdirectly.Inthirdcase,Ilteredimagewitha5x5gaussiankernel
toremovethenoise,thenappliedOtsuthresholding.Seehownoiselteringimprovestheresult.
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

noisy2.png

,
0
)
#globalthresholding
Result:
56Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
HowOtsu'sBinarizationWorks?
ThissectiondemonstratesaPythonimplementationofOtsu'sbinarizationtoshowhowitworksactually.Ifyouare
notinterested,youcanskipthis.
Sinceweareworkingwithbimodalimages,Otsu'salgorithmtriestondathresholdvalue(t)whichminimizesthe
weightedwithin-classvariance
givenbytherelation:

2
w
(
t
)=
q
1
(
t
)

2
1
(
t
)+
q
2
(
t
)

2
2
(
t
)
where
q
1
(
t
)=
t
X
i
=1
P
(
i
)&
q
1
(
t
)=
I
X
i
=
t
+1
P
(
i
)

1
(
t
)=
t
X
i
=1
iP
(
i
)
q
1
(
t
)
&

2
(
t
)=
I
X
i
=
t
+1
iP
(
i
)
q
2
(
t
)

2
1
(
t
)=
t
X
i
=1
[
i


1
(
t
)]
2
P
(
i
)
q
1
(
t
)
&

2
2
(
t
)=
I
X
i
=
t
+1
[
i


1
(
t
)]
2
P
(
i
)
q
2
(
t
)
img
=
cv2
.
imread(

noisy2.png

,
0
)
blur
=
cv2
.
GaussianBlur(img,(
5
,
5
),
0
)
#findnormalized_histogram,anditscumulativedistributionfunction
1.4.ImageProcessinginOpenCV57
OpenCV-PythonTutorialsDocumentation,Release1
hist
=
cv2
.
calcHist([blur],[
0
],
None
,[
256
],[
0
,
256
])
hist_norm
=
hist
.
ravel()
/
hist
.
max()
Q
=
hist_norm
.
cumsum()
bins
=
np
.
arange(
256
)
fn_min
=
np
.
inf
thresh
=
-
1
for
i
in
xrange(
1
,
256
):
p1,p2
=
np
.
hsplit(hist_norm,[i])
#probabilities
q1,q2
=
Q[i],Q[
255
]
-
Q[i]
#cumsumofclasses
b1,b2
=
np
.
hsplit(bins,[i])
#weights
#findingmeansandvariances
m1,m2
=
np
.
sum(p1
*
b1)
/
q1,np
.
sum(p2
*
b2)
/
q2
v1,v2
=
np
.
sum(((b1
-
m1)
*
*
2
)
*
p1)
/
q1,np
.
sum(((b2
-
m2)
*
*
2
)
*
p2)
/
q2
#calculatestheminimizationfunction
fn
=
v1
*
q1
+
v2
*
q2
if
fn

fn_min:
fn_min
=
fn
thresh
=
i
#findotsusthresholdvaluewithOpenCVfunction
(Someofthefunctionsmaybenewhere,butwewillcoverthemincomingchapters)
AdditionalResources
1.
DigitalImageProcessing,RafaelC.Gonzalez
Exercises
1.
TherearesomeoptimizationsavailableforOtsu'sbinarization.Youcansearchandimplementit.
58Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Scaling
Scalingisjustresizingoftheimage.OpenCVcomeswithafunction
cv2.resize()
forthispurpose.Thesizeof
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

messi5.jpg

)
res
=
cv2
.
resize(img,
None
,fx
=
2
,fy
=
2
,interpolation
=
cv2
.
INTER_CUBIC)
#OR
height,width
=
img
.
shape[:
2
]
res
=
cv2
.
resize(img,(
2
*
width,
2
*
height),interpolation
=
cv2
.
INTER_CUBIC)
Translation
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

messi5.jpg

,
0
)
rows,cols
=
img
.
shape
M
=
np
.
float32([[
1
,
0
,
100
],[
0
,
1
,
50
]])
dst
=
cv2
.
warpAffine(img,M,(cols,rows))
cv2
.
imshow(

img

,dst)
cv2
.
waitKey(
0
)
cv2
.
destroyAllWindows()
Warning:
Thirdargumentofthe
cv2.warpAfne()
functionisthesizeoftheoutputimage,whichshouldbein
theformof
(width,height)
.Rememberwidth=numberofcolumns,andheight=numberofrows.
Seetheresultbelow:
1.4.ImageProcessinginOpenCV59
OpenCV-PythonTutorialsDocumentation,Release1
Rotation
Rotationofanimageforanangle

isachievedbythetransformationmatrixoftheform
M
=

cos

sin
sincos

ButOpenCVprovidesscaledrotationwithadjustablecenterofrotationsothatyoucanrotateatanylocationyou
prefer.Modiedtransformationmatrixisgivenby


(1


)

center:x



center:y



center:x
+(1


)

center:y

where:

=
scale

cos
;

=
scale

sin

Tondthistransformationmatrix,OpenCVprovidesafunction,
img
=
cv2
.
imread(

messi5.jpg

,
0
)
rows,cols
=
img
.
shape
M
=
cv2
.
Seetheresult:
60Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AfneTransformation
Inafnetransformation,allparallellinesintheoriginalimagewillstillbeparallelintheoutputimage.Tondthe
transformationmatrix,weneedthreepointsfrominputimageandtheircorrespondinglocationsinoutputimage.Then
img
=
cv2
.
imread(

drawing.png

)
rows,cols,ch
=
img
.
shape
pts1
=
np
.
float32([[
50
,
50
],[
200
,
50
],[
50
,
200
]])
pts2
=
np
.
float32([[
10
,
100
],[
200
,
50
],[
100
,
250
]])
M
=
cv2
.
Seetheresult:
PerspectiveTransformation
Forperspectivetransformation,youneeda3x3transformationmatrix.Straightlineswillremainstraightevenafter
thetransformation.Tondthistransformationmatrix,youneed4pointsontheinputimageandcorrespondingpoints
ontheoutputimage.Amongthese4points,3ofthemshouldnotbecollinear.Thentransformationmatrixcanbe
foundbythefunction
1.4.ImageProcessinginOpenCV61
OpenCV-PythonTutorialsDocumentation,Release1
img
=
cv2
.
imread(

sudokusmall.png

)
rows,cols,ch
=
img
.
shape
pts1
=
np
.
float32([[
56
,
65
],[
368
,
52
],[
28
,
387
],[
389
,
390
]])
pts2
=
np
.
float32([[
0
,
0
],[
300
,
0
],[
0
,
300
],[
300
,
300
]])
M
=
cv2
.
Result:
AdditionalResources
1.
“ComputerVision:AlgorithmsandApplications”,RichardSzeliski
Exercises
SmoothingImages
Goals
Learnto:

Blurimagesswithvariouslowpasslters

Applycustom-madelterstoimages(2Dconvolution)
62Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
2DConvolution(ImageFiltering)
Asforone-dimensionalsignals,imagesalsocanbelteredwithvariouslow-passlters(LPF),high-passlters(HPF),
25
2
6
6
6
6
4
11111
11111
11111
11111
11111
3
7
7
7
7
5
Filteringwiththeabovekernelresultsinthefollowingbeingperformed:foreachpixel,a5x5windowiscenteredon
thispixel,allpixelsfallingwithinthiswindowaresummedup,andtheresultisthendividedby25.Thisequatesto
computingtheaverageofthepixelvaluesinsidethatwindow.Thisoperationisperformedforallthepixelsinthe
imagetoproducetheoutputlteredimage.Trythiscodeandchecktheresult:
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

opencv_logo.png

)
kernel
=
np
.
ones((
5
,
5
),np
.
float32)
/
25
dst
=
cv2
.
filter2D(img,
-
1
,kernel)
plt
.
subplot(
121
),plt
.
imshow(img),plt
.
title(

Original

)
plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
122
),plt
.
imshow(dst),plt
.
title(

Averaging

)
plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Result:
1.4.ImageProcessinginOpenCV63
OpenCV-PythonTutorialsDocumentation,Release1
ImageBlurring(ImageSmoothing)
Imageblurringisachievedbyconvolvingtheimagewithalow-passlterkernel.Itisusefulforremovingnoise.It
actuallyremoveshighfrequencycontent(e.g:noise,edges)fromtheimageresultinginedgesbeingblurredwhenthis
islterisapplied.(Well,thereareblurringtechniqueswhichdonotbluredges).OpenCVprovidesmainlyfourtypes
ofblurringtechniques.
1.Averaging
Thisisdonebyconvolvingtheimagewithanormalizedboxlter.Itsimplytakestheaverageofallthepixels
underkernelareaandreplacesthecentralelementwiththisaverage.Thisisdonebythefunction
cv2.blur()
or
cv2.boxFilter()
9
2
4
111
111
111
3
5
Note:
Ifyoudon'twanttouseanormalizedboxlter,use
cv2.boxFilter()
andpasstheargument
normalize=False
tothefunction.
Checkthesampledemobelowwithakernelof5x5size:
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

opencv_logo.png

)
64Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
blur
=
cv2
.
blur(img,(
5
,
5
))
plt
.
subplot(
121
),plt
.
imshow(img),plt
.
title(

Original

)
plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
122
),plt
.
imshow(blur),plt
.
title(

Blurred

)
plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Result:
2.GaussianFiltering
Inthisapproach,insteadofaboxlterconsistingofequalltercoefcients,aGaussiankernelisused.Itisdonewith
thefunction,
cv2.GaussianBlur()
.Weshouldspecifythewidthandheightofthekernelwhichshouldbepositiveand
odd.WealsoshouldspecifythestandarddeviationintheXandYdirections,sigmaXandsigmaYrespectively.If
onlysigmaXisspecied,sigmaYistakenasequaltosigmaX.Ifbotharegivenaszeros,theyarecalculatedfromthe
kernelsize.GaussianlteringishighlyeffectiveinremovingGaussiannoisefromtheimage.
Ifyouwant,youcancreateaGaussiankernelwiththefunction,
blur
=
cv2
.
GaussianBlur(img,(
5
,
5
),
0
)
Result:
1.4.ImageProcessinginOpenCV65
OpenCV-PythonTutorialsDocumentation,Release1
3.MedianFiltering
Here,thefunction
cv2.medianBlur()
computesthemedianofallthepixelsunderthekernelwindowandthecentral
pixelisreplacedwiththismedianvalue.Thisishighlyeffectiveinremovingsalt-and-peppernoise.Oneinteresting
thingtonoteisthat,intheGaussianandboxlters,thelteredvalueforthecentralelementcanbeavaluewhichmay
notexistintheoriginalimage.Howeverthisisnotthecaseinmedianltering,sincethecentralelementisalways
replacedbysomepixelvalueintheimage.Thisreducesthenoiseeffectively.Thekernelsizemustbeapositiveodd
integer.
Inthisdemo,weadda50%noisetoouroriginalimageanduseamedianlter.Checktheresult:
median
=
cv2
.
medianBlur(img,
5
)
Result:
66Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
4.BilateralFiltering
Aswenoted,thelterswepresentedearliertendtobluredges.Thisisnotthecaseforthebilaterallter,
cv2.bilateralFilter()
,whichwasdenedfor,andishighlyeffectiveatnoiseremovalwhilepreservingedges.But
theoperationisslowercomparedtootherlters.WealreadysawthataGaussianltertakestheaneighborhood
aroundthepixelandndsitsGaussianweightedaverage.ThisGaussianlterisafunctionofspacealone,thatis,
blur
=
cv2
.
bilateralFilter(img,
9
,
75
,
75
)
Result:
1.4.ImageProcessinginOpenCV67
OpenCV-PythonTutorialsDocumentation,Release1
Notethatthetextureonthesurfaceisgone,butedgesarestillpreserved.
AdditionalResources
1.
68Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
1.Erosion
Thebasicideaoferosionisjustlikesoilerosiononly,iterodesawaytheboundariesofforegroundobject(Always
trytokeepforegroundinwhite).Sowhatdoesitdo?Thekernelslidesthroughtheimage(asin2Dconvolution).A
pixelintheoriginalimage(either1or0)willbeconsidered1onlyifallthepixelsunderthekernelis1,otherwiseit
iseroded(madetozero).
Sowhathappendsisthat,allthepixelsnearboundarywillbediscardeddependinguponthesizeofkernel.Sothe
thicknessorsizeoftheforegroundobjectdecreasesorsimplywhiteregiondecreasesintheimage.Itisusefulfor
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

j.png

,
0
)
kernel
=
np
.
ones((
5
,
5
),np
.
uint8)
erosion
=
cv2
.
erode(img,kernel,iterations
=
1
)
Result:
2.Dilation
Itisjustoppositeoferosion.Here,apixelelementis`1'ifatleastonepixelunderthekernelis`1'.Soitincreases
thewhiteregionintheimageorsizeofforegroundobjectincreases.Normally,incaseslikenoiseremoval,erosion
1.4.ImageProcessinginOpenCV69
OpenCV-PythonTutorialsDocumentation,Release1
isfollowedbydilation.Because,erosionremoveswhitenoises,butitalsoshrinksourobject.Sowedilateit.Since
noiseisgone,theywon'tcomeback,butourobjectareaincreases.Itisalsousefulinjoiningbrokenpartsofanobject.
dilation
=
cv2
.
dilate(img,kernel,iterations
=
1
)
Result:
3.Opening
Openingisjustanothernameof
erosionfollowedbydilation
.Itisusefulinremovingnoise,asweexplainedabove.
Hereweusethefunction,
cv2.morphologyEx()
opening
=
cv2
.
morphologyEx(img,cv2
.
MORPH_OPEN,kernel)
Result:
4.Closing
ClosingisreverseofOpening,
DilationfollowedbyErosion
.Itisusefulinclosingsmallholesinsidetheforeground
objects,orsmallblackpointsontheobject.
closing
=
cv2
.
morphologyEx(img,cv2
.
MORPH_CLOSE,kernel)
Result:
70Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
5.MorphologicalGradient
gradient
=
cv2
.
morphologyEx(img,cv2
.
MORPH_GRADIENT,kernel)
Result:
6.TopHat
tophat
=
cv2
.
morphologyEx(img,cv2
.
MORPH_TOPHAT,kernel)
Result:
1.4.ImageProcessinginOpenCV71
OpenCV-PythonTutorialsDocumentation,Release1
7.BlackHat
blackhat
=
cv2
.
morphologyEx(img,cv2
.
MORPH_BLACKHAT,kernel)
Result:
StructuringElement
WemanuallycreatedastructuringelementsinthepreviousexampleswithhelpofNumpy.Itisrectangularshape.
Butinsomecases,youmayneedelliptical/circularshapedkernels.Soforthispurpose,OpenCVhasafunction,
#RectangularKernel
��

cv2
.
72Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
[
1
,
1
,
1
,
1
,
1
],
[
0
,
0
,
1
,
0
,
0
]],dtype
=
uint8)
#Cross-shapedKernel
��

cv2
.
AdditionalResources
1.
MorphologicalOperations
atHIPR2
Exercises
ImageGradients
Goal
Inthischapter,wewilllearnto:

@x
2
+
@
2
src
@y
2
whereeachderivativeisfound
usingSobelderivatives.If
ksize=1
,thenfollowingkernelisusedforltering:
kernel
=
2
4
010
1

41
010
3
5
1.4.ImageProcessinginOpenCV73
OpenCV-PythonTutorialsDocumentation,Release1
Code
Belowcodeshowsalloperatorsinasinglediagram.Allkernelsareof5x5size.Depthofoutputimageispassed-1to
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

dave.jpg

,
0
)
laplacian
=
cv2
.
Laplacian(img,cv2
.
CV_64F)
sobelx
=
cv2
.
Sobel(img,cv2
.
CV_64F,
1
,
0
,ksize
=
5
)
sobely
=
cv2
.
Sobel(img,cv2
.
CV_64F,
0
,
1
,ksize
=
5
)
plt
.
subplot(
2
,
2
,
1
),plt
.
imshow(img,cmap
=

gray

)
plt
.
title(

Original

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
2
,
2
,
2
),plt
.
imshow(laplacian,cmap
=

gray

)
plt
.
title(

Laplacian

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
2
,
2
,
3
),plt
.
imshow(sobelx,cmap
=

gray

)
plt
.
title(

SobelX

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
2
,
2
,
4
),plt
.
imshow(sobely,cmap
=

gray

)
plt
.
title(

SobelY

),plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Result:
74Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
OneImportantMatter!
Inourlastexample,outputdatatypeiscv2.CV_8Uornp.uint8.Butthereisaslightproblemwiththat.Black-to-White
transitionistakenasPositiveslope(ithasapositivevalue)whileWhite-to-BlacktransitionistakenasaNegativeslope
(Ithasnegativevalue).Sowhenyouconvertdatatonp.uint8,allnegativeslopesaremadezero.Insimplewords,you
missthatedge.
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
1.4.ImageProcessinginOpenCV75
OpenCV-PythonTutorialsDocumentation,Release1
img
=
cv2
.
imread(

box.png

,
0
)
#Outputdtype=cv2.CV_8U
sobelx8u
=
cv2
.
Sobel(img,cv2
.
CV_8U,
1
,
0
,ksize
=
5
)
#Outputdtype=cv2.CV_64F.Thentakeitsabsoluteandconverttocv2.CV_8U
sobelx64f
=
cv2
.
Sobel(img,cv2
.
CV_64F,
1
,
0
,ksize
=
5
)
abs_sobel64f
=
np
.
absolute(sobelx64f)
sobel_8u
=
np
.
uint8(abs_sobel64f)
plt
.
subplot(
1
,
3
,
1
),plt
.
imshow(img,cmap
=

gray

)
plt
.
title(

Original

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
1
,
3
,
2
),plt
.
imshow(sobelx8u,cmap
=

gray

)
plt
.
title(

SobelCV_8U

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
1
,
3
,
3
),plt
.
imshow(sobel_8u,cmap
=

gray

)
plt
.
title(

Sobelabs(CV_64F)

),plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Checktheresultbelow:
AdditionalResources
Exercises
76Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
G
2
x
+
G
2
y
Angle
(

)=tan

1

G
y
G
x

Gradientdirectionisalwaysperpendiculartoedges.Itisroundedtooneoffouranglesrepresentingvertical,horizontal
andtwodiagonaldirections.
3.
Non-maximumSuppression
PointAisontheedge(inverticaldirection).Gradientdirectionisnormaltotheedge.PointBandCareingradient
directions.SopointAischeckedwithpointBandCtoseeifitformsalocalmaximum.Ifso,itisconsideredfornext
stage,otherwise,itissuppressed(puttozero).
1.4.ImageProcessinginOpenCV77
OpenCV-PythonTutorialsDocumentation,Release1
TheedgeAisabovethe
maxVal
,soconsideredas“sure-edge”.AlthoughedgeCisbelow
maxVal
,itisconnectedto
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

messi5.jpg

,
0
)
edges
=
cv2
.
Canny(img,
100
,
200
)
plt
.
subplot(
121
),plt
.
imshow(img,cmap
=

gray

)
plt
.
title(

OriginalImage

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
122
),plt
.
imshow(edges,cmap
=

gray

)
plt
.
title(

EdgeImage

),plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Seetheresultbelow:
78Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
1.4.ImageProcessinginOpenCV79
OpenCV-PythonTutorialsDocumentation,Release1
img
=
cv2
.
imread(

messi5.jpg

)
lower_reso
=
cv2
.
pyrDown(higher_reso)
Belowisthe4levelsinanimagepyramid.
Nowyoucangodowntheimagepyramidwith
cv2.pyrUp()
function.
higher_reso2
=
cv2
.
pyrUp(lower_reso)
Remember,
higher_reso2
isnotequalto
higher_reso
,becauseonceyoudecreasetheresolution,youloosetheinfor-
mation.Belowimageis3leveldownthepyramidcreatedfromsmallestimageinpreviouscase.Compareitwith
originalimage:
80Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
LaplacianPyramidsareformedfromtheGaussianPyramids.Thereisnoexclusivefunctionforthat.Laplacian
pyramidimagesarelikeedgeimagesonly.Mostofitselementsarezeros.Theyareusedinimagecompression.A
1.4.ImageProcessinginOpenCV81
OpenCV-PythonTutorialsDocumentation,Release1
82Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
ImageBlendingusingPyramids
OneapplicationofPyramidsisImageBlending.Forexample,inimagestitching,youwillneedtostacktwoimages
1.4.ImageProcessinginOpenCV83
OpenCV-PythonTutorialsDocumentation,Release1
Belowisthefullcode.(Forsakeofsimplicity,eachstepisdoneseparatelywhichmaytakemorememory.Youcan
optimizeitifyouwantso).
import
cv2
import
numpy
as
np
,
sys
A
=
cv2
.
imread(

apple.jpg

)
B
=
cv2
.
imread(

orange.jpg

)
#generateGaussianpyramidforA
G
=
A
.
copy()
gpA
=
[G]
for
i
in
xrange(
6
):
G
=
cv2
.
pyrDown(G)
gpA
.
append(G)
#generateGaussianpyramidforB
G
=
B
.
copy()
gpB
=
[G]
for
i
in
xrange(
6
):
G
=
cv2
.
pyrDown(G)
gpB
.
append(G)
#generateLaplacianPyramidforA
lpA
=
[gpA[
5
]]
for
i
in
xrange(
5
,
0
,
-
1
):
GE
=
cv2
.
pyrUp(gpA[i])
L
=
cv2
.
subtract(gpA[i
-
1
],GE)
lpA
.
append(L)
#generateLaplacianPyramidforB
lpB
=
[gpB[
5
]]
for
i
in
xrange(
5
,
0
,
-
1
):
GE
=
cv2
.
pyrUp(gpB[i])
L
=
cv2
.
subtract(gpB[i
-
1
],GE)
lpB
.
append(L)
#Nowaddleftandrighthalvesofimagesineachlevel
LS
=
[]
for
la,lb
in
zip
(lpA,lpB):
rows,cols,dpt
=
la
.
shape
ls
=
np
.
hstack((la[:,
0
:cols
/
2
],lb[:,cols
/
2
:]))
LS
.
append(ls)
#nowreconstruct
ls_
=
LS[
0
]
for
i
in
xrange(
1
,
6
):
ls_
=
cv2
.
pyrUp(ls_)
ls_
=
cv2
.
add(ls_,LS[i])
#imagewithdirectconnectingeachhalf
real
=
np
.
hstack((A[:,:cols
/
2
],B[:,cols
/
2
:]))
cv2
.
imwrite(

Pyramid_blending2.jpg

,ls_)
cv2
.
imwrite(

Direct_blending.jpg

,real)
84Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
ImageBlending
Exercises
ContoursinOpenCV

LearntondanddrawContours

ContourFeatures

ContourProperties
LearntonddifferentpropertiesofcontourslikeSolidity,MeanIntensity

Contours:MoreFunctions
Learntondconvexitydefects,pointPolygonTest,matchdifferentshapes

ContoursHierarchy
1.4.ImageProcessinginOpenCV85
OpenCV-PythonTutorialsDocumentation,Release1
LearnaboutContourHierarchy
86Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
import
numpy
as
np
import
cv2
im
=
cv2
.
imread(

test.jpg

)
imgray
=
cv2
.
cvtColor(im,cv2
.
COLOR_BGR2GRAY)
See,therearethreeargumentsin
cv2.ndContours()
Note:
Howtodrawthecontours?
Todrawthecontours,
cv2.drawContours
functionisused.Itcanalsobeusedtodrawanyshapeprovidedyou
haveitsboundarypoints.Itsrstargumentissourceimage,secondargumentisthecontourswhichshouldbepassed
asaPythonlist,thirdargumentisindexofcontours(usefulwhendrawingindividualcontour.Todrawallcontours,
img
=
cv2
.
drawContours(img,contours,
-
1
,(
0
,
255
,
0
),
3
)
Todrawanindividualcontour,say4thcontour:
1.4.ImageProcessinginOpenCV87
OpenCV-PythonTutorialsDocumentation,Release1
img
=
cv2
.
drawContours(img,contours,
3
,(
0
,
255
,
0
),
3
)
cnt
=
contours[
4
]
img
=
cv2
.
drawContours(img,[cnt],
0
,(
0
,
255
,
0
),
3
)
Note:
AdditionalResources
Exercises
ContourFeatures
Goal
Inthisarticle,wewilllearn

88Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
1.Moments
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

star.jpg

,
0
)
M
00
and
C
y
=
M
01
M
00
.Thiscanbedoneasfollows:
cx
=
int
(M[

m10

]
/
M[

m00

])
cy
=
int
(M[

m01

]
/
M[

m00

])
2.ContourArea
Contourareaisgivenbythefunction
cv2.contourArea()
orfrommoments,
M['m00']
.
area
=
cv2
.
contourArea(cnt)
4.ContourApproximation
Itapproximatesacontourshapetoanothershapewithlessnumberofverticesdependingupontheprecisionwespecify.
Itisanimplementationof
Douglas-Peuckeralgorithm
.Checkthewikipediapageforalgorithmanddemonstration.
Tounderstandthis,supposeyouaretryingtondasquareinanimage,butduetosomeproblemsintheimage,you
epsilon
=
0.1
*
cv2
.
arcLength(cnt,
True
)
approx
=
cv2
.
approxPolyDP(cnt,epsilon,
True
)
Below,insecondimage,greenlineshowstheapproximatedcurvefor
epsilon=10%ofarclength
.Third
imageshowsthesamefor
epsilon=1%ofthearclength
1.4.ImageProcessinginOpenCV89
OpenCV-PythonTutorialsDocumentation,Release1
5.ConvexHull
ConvexHullwilllooksimilartocontourapproximation,butitisnot(Bothmayprovidesameresultsinsomecases).
Here,
cv2.convexHull()
functionchecksacurveforconvexitydefectsandcorrectsit.Generallyspeaking,convex
curvesarethecurveswhicharealwaysbulgedout,orat-leastat.Andifitisbulgedinside,itiscalledconvexity
defects.Forexample,checkthebelowimageofhand.Redlineshowstheconvexhullofhand.Thedouble-sided
arrowmarksshowstheconvexitydefects,whicharethelocalmaximumdeviationsofhullfromcontours.
Thereisalittlebitthingstodiscussaboutititssyntax:
hull
=
cv2
.
90Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1

clockwise
:Orientationag.Ifitis
True
,theoutputconvexhullisorientedclockwise.Otherwise,itisoriented
counter-clockwise.

hull
=
cv2
.
convexHull(cnt)
Butifyouwanttondconvexitydefects,youneedtopass
k
=
cv2
.
isContourConvex(cnt)
7.BoundingRectangle
Therearetwotypesofboundingrectangles.
7.a.StraightBoundingRectangle
Itisastraightrectangle,itdoesn'tconsidertherotationoftheobject.Soareaoftheboundingrectanglewon'tbe
minimum.Itisfoundbythefunction
cv2.boundingRect()
.
x,y,w,h
=
cv2
.
boundingRect(cnt)
img
=
cv2
.
rectangle(img,(x,y),(x
+
w,y
+
h),(
0
,
255
,
0
),
2
)
7.b.RotatedRectangle
Here,boundingrectangleisdrawnwithminimumarea,soitconsiderstherotationalso.Thefunctionusedis
cv2.minAreaRect()
function
cv2.boxPoints()
rect
=
cv2
.
minAreaRect(cnt)
box
=
cv2
.
boxPoints(rect)
box
=
np
.
int0(box)
im
=
cv2
.
drawContours(im,[box],
0
,(
0
,
0
,
255
),
2
)
1.4.ImageProcessinginOpenCV91
OpenCV-PythonTutorialsDocumentation,Release1
Boththerectanglesareshowninasingleimage.Greenrectangleshowsthenormalboundingrect.Redrectangleis
therotatedrect.
8.MinimumEnclosingCircle
Nextwendthecircumcircleofanobjectusingthefunction
cv2.minEnclosingCircle()
.Itisacirclewhichcom-
(x,y),radius
=
cv2
.
minEnclosingCircle(cnt)
center
=
(
int
(x),
int
(y))
radius
=
int
(radius)
img
=
cv2
.
circle(img,center,radius,(
0
,
255
,
0
),
2
)
92Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
9.FittinganEllipse
ellipse
=
cv2
.
fitEllipse(cnt)
im
=
cv2
.
ellipse(im,ellipse,(
0
,
255
,
0
),
2
)
1.4.ImageProcessinginOpenCV93
OpenCV-PythonTutorialsDocumentation,Release1
10.FittingaLine
rows,cols
=
img
.
shape[:
2
]
[vx,vy,x,y]
=
cv2
.
fitLine(cnt,cv2
.
DIST_L2,
0
,
0.01
,
0.01
)
lefty
=
int
((
-
x
*
vy
/
vx)
+
y)
righty
=
int
(((cols
-
x)
*
vy
/
vx)
+
y)
img
=
cv2
.
line(img,(cols
-
1
,righty),(
0
,lefty),(
0
,
255
,
0
),
2
)
94Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
ContourProperties
Height
x,y,w,h
=
cv2
.
boundingRect(cnt)
aspect_ratio
=
float
(w)
/
h
2.Extent
Extentistheratioofcontourareatoboundingrectanglearea.
Extent
=
ObjectArea
BoundingRectangleArea
area
=
cv2
.
contourArea(cnt)
x,y,w,h
=
cv2
.
boundingRect(cnt)
rect_area
=
w
*
h
extent
=
float
(area)
/
rect_area
1.4.ImageProcessinginOpenCV95
OpenCV-PythonTutorialsDocumentation,Release1
3.Solidity
Solidityistheratioofcontourareatoitsconvexhullarea.
Solidity
=
ContourArea
ConvexHullArea
area
=
cv2
.
contourArea(cnt)
hull
=
cv2
.
convexHull(cnt)
hull_area
=
cv2
.
contourArea(hull)
solidity
=
float
(area)
/
hull_area
4

ContourArea

area
=
cv2
.
contourArea(cnt)
5.Orientation
(x,y),(MA,ma),angle
=
cv2
.
fitEllipse(cnt)
6.MaskandPixelPoints
Insomecases,wemayneedallthepointswhichcomprisesthatobject.Itcanbedoneasfollows:
mask
=
np
.
zeros(imgray
.
shape,np
.
uint8)
cv2
.
drawContours(mask,[cnt],
0
,
255
,
-
1
)
pixelpoints
=
np
.
transpose(np
.
nonzero(mask))
#pixelpoints=cv2.findNonZero(mask)
min_val,max_val,min_loc,max_loc
=
cv2
.
minMaxLoc(imgray,mask
=
mask)
96Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
8.MeanColororMeanIntensity
Here,wecanndtheaveragecolorofanobject.Oritcanbeaverageintensityoftheobjectingrayscalemode.We
againusethesamemasktodoit.
mean_val
=
cv2
.
mean(im,mask
=
mask)
9.ExtremePoints
ExtremePointsmeanstopmost,bottommost,rightmostandleftmostpointsoftheobject.
leftmost
=
tuple
(cnt[cnt[:,:,
0
]
.
argmin()][
0
])
rightmost
=
tuple
(cnt[cnt[:,:,
0
]
.
argmax()][
0
])
topmost
=
tuple
(cnt[cnt[:,:,
1
]
.
argmin()][
0
])
bottommost
=
tuple
(cnt[cnt[:,:,
1
]
.
argmax()][
0
])
1.4.ImageProcessinginOpenCV97
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
1.
Therearestillsomefeaturesleftinmatlabregionpropsdoc.Trytoimplementthem.
Contours:MoreFunctions
Goal
Inthischapter,wewilllearnabout

Convexitydefectsandhowtondthem.

Findingshortestdistancefromapointtoapolygon
98Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1

Matchingdifferentshapes
TheoryandCode
1.ConvexityDefects
Wesawwhatisconvexhullinsecondchapteraboutcontours.Anydeviationoftheobjectfromthishullcanbe
consideredasconvexitydefect.
OpenCVcomeswithaready-madefunctiontondthis,
cv2.convexityDefects()
.Abasicfunctioncallwouldlook
likebelow:
hull
=
cv2
.
Note:
Rememberwehavetopass
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

star.jpg

)
img_gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
Andseetheresult:
1.4.ImageProcessinginOpenCV99
OpenCV-PythonTutorialsDocumentation,Release1
2.PointPolygonTest
dist
=
cv2
.
pointPolygonTest(cnt,(
50
,
50
),
True
)
Inthefunction,thirdargumentis
measureDist
.Ifitis
True
,itndsthesigneddistance.If
False
,itnds
Note:
Ifyoudon'twanttondthedistance,makesurethirdargumentis
False
,because,itisatimeconsuming
process.So,makingit
False
givesabout2-3Xspeedup.
3.MatchShapes
OpenCVcomeswithafunction
cv2.matchShapes()
whichenablesustocomparetwoshapes,ortwocontoursand
import
cv2
import
numpy
as
np
img1
=
cv2
.
imread(

star.jpg

,
0
)
img2
=
cv2
.
imread(

star2.jpg

,
0
)
100Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Itriedmatchingshapeswithdifferentshapesgivenbelow:
Igotfollowingresults:

MatchingImageAwithitself=0.0

MatchingImageAwithImageB=0.001946

MatchingImageAwithImageC=0.326911
See,evenimagerotationdoesn'taffectmuchonthiscomparison.
Seealso:
Hu-Moments
aresevenmomentsinvarianttotranslation,rotationandscale.Seventhoneisskew-invariant.Those
valuescanbefoundusing
cv2.HuMoments()
function.
AdditionalResources
Exercises
1.
Checkthedocumentationfor
cv2.pointPolygonTest()
,youcanndaniceimageinRedandBluecolor.It
representsthedistancefromallpixelstothewhitecurveonit.Allpixelsinsidecurveisbluedependingonthe
distance.Similarlyoutsidepointsarered.ContouredgesaremarkedwithWhite.Soproblemissimple.Write
acodetocreatesucharepresentationofdistance.
2.
1.4.ImageProcessinginOpenCV101
OpenCV-PythonTutorialsDocumentation,Release1
Also,intheoutput,wegotthreearrays,rstistheimage,secondisourcontours,andonemoreoutputwhichwe
namedas
hierarchy
(Pleasecheckoutthecodesinpreviousarticles).Butweneverusedthishierarchyanywhere.
Thenwhatisthishierarchyandwhatisitfor?Whatisitsrelationshipwiththepreviousmentionedfunctionargument
?
Thatiswhatwearegoingtodealinthisarticle.
WhatisHierarchy?
Normallyweusethe
cv2.ndContours()
Inthisimage,thereareafewshapeswhichIhavenumberedfrom
0-5
.
2and2a
denotestheexternalandinternal
contoursoftheoutermostbox.
Here,contours0,1,2are
externaloroutermost
.Wecansay,theyarein
hierarchy-0
orsimplytheyarein
same
hierarchylevel
.
Nextcomes
contour-2a
.Itcanbeconsideredasa
childofcontour-2
(orinoppositeway,contour-2isparentof
102Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
HierarchyRepresentationinOpenCV
Note:
Ifthereisnochildorparent,thateldistakenas-1
���
hierarchy
array([[[1,-1,-1,-1],
[2,0,-1,-1],
[3,1,-1,-1],
[4,2,-1,-1],
[5,3,-1,-1],
[6,4,-1,-1],
1.4.ImageProcessinginOpenCV103
OpenCV-PythonTutorialsDocumentation,Release1
[7,5,-1,-1],
[-1,6,-1,-1]]])
Thisisthegoodchoicetouseinyourcode,ifyouarenotusinganyhierarchyfeatures.
���
hierarchy
array([[[1,-1,-1,-1],
[2,0,-1,-1],
[-1,1,-1,-1]]])
Youcanusethisagifyouwanttoextractonlytheoutercontours.Itmightbeusefulinsomecases.
104Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Soconsiderrstcontour,iecontour-0.Itishierarchy-1.Ithastwoholes,contours1&2,andtheybelongtohierarchy-
2.Soforcontour-0,Nextcontourinsamehierarchyleveliscontour-3.Andthereisnopreviousone.Anditsrstis
childiscontour-1inhierarchy-2.Ithasnoparent,becauseitisinhierarchy-1.Soitshierarchyarrayis[3,-1,1,-1]
Nowtakecontour-1.Itisinhierarchy-2.Nextoneinsamehierarchy(undertheparenthoodofcontour-1)iscontour-2.
Nopreviousone.Nochild,butparentiscontour-0.Soarrayis[2,-1,-1,0].
Similarlycontour-2:Itisinhierarchy-2.Thereisnotnextcontourinsamehierarchyundercontour-0.SonoNext.
Previousiscontour-1.Nochild,parentiscontour-0.Soarrayis[-1,1,-1,0].
Contour-3:Nextinhierarchy-1iscontour-5.Previousiscontour-0.Childiscontour-4andnoparent.Soarrayis
[5,0,4,-1].
Contour-4:Itisinhierarchy2undercontour-3andithasnosibling.Sononext,noprevious,nochild,parentis
contour-3.Soarrayis[-1,-1,-1,3].
Remainingyoucanllup.ThisisthenalanswerIgot:
���
hierarchy
array([[[3,-1,1,-1],
[2,-1,-1,0],
[-1,1,-1,0],
[5,0,4,-1],
[-1,-1,-1,3],
[7,3,6,-1],
[-1,-1,-1,5],
[8,5,-1,-1],
[-1,7,-1,-1]]])
1.4.ImageProcessinginOpenCV105
OpenCV-PythonTutorialsDocumentation,Release1
Takecontour-0:Itisinhierarchy-0.Nextcontourinsamehierarchyiscontour-7.Nopreviouscontours.Childis
contour-1.Andnoparent.Soarrayis[7,-1,1,-1].
Takecontour-2:Itisinhierarchy-1.Nocontourinsamelevel.Nopreviousone.Childiscontour-2.Parentis
contour-0.Soarrayis[-1,-1,2,0].
Andremaining,tryyourself.Belowisthefullanswer:
���
hierarchy
array([[[7,-1,1,-1],
[-1,-1,2,0],
[-1,-1,3,1],
[-1,-1,4,2],
[-1,-1,5,3],
[6,-1,-1,4],
[-1,5,-1,4],
[8,0,-1,-1],
[-1,7,-1,-1]]])
AdditionalResources
106Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Exercises
HistogramsinOpenCV

Histograms-1:Find,Plot,Analyze!!!
LearntondanddrawContours

Histograms-2:HistogramEqualization

Histograms-3:2DHistograms
Learntondandplot2DHistograms

Histogram-4:HistogramBackprojection
Learnhistogrambackprojectiontosegmentcoloredobjects
1.4.ImageProcessinginOpenCV107
OpenCV-PythonTutorialsDocumentation,Release1
Histograms-1:Find,Plot,Analyze!!!
Goal
Learnto

Findhistograms,usingbothOpenCVandNumpyfunctions

Plothistograms,usingOpenCVandMatplotlibfunctions

Youwillseethesefunctions:
cv2.calcHist()
,
np.histogram()
Youcanseetheimageanditshistogram.(Remember,thishistogramisdrawnforgrayscaleimage,notcolorimage).
Leftregionofhistogramshowstheamountofdarkerpixelsinimageandrightregionshowstheamountofbrighter
pixels.Fromthehistogram,youcanseedarkregionismorethanbrighterregion,andamountofmidtones(pixel
valuesinmid-range,sayaround127)areveryless.
108Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
FindHistogram
Nowwehaveanideaonwhatishistogram,wecanlookintohowtondthis.BothOpenCVandNumpycome
within-builtfunctionforthis.Beforeusingthosefunctions,weneedtounderstandsometerminologiesrelatedwith
histograms.
BINS
:Theabovehistogramshowsthenumberofpixelsforeverypixelvalue,iefrom0to255.ieyouneed256
valuestoshowtheabovehistogram.Butconsider,whatifyouneednotndthenumberofpixelsforallpixelvalues
separately,butnumberofpixelsinaintervalofpixelvalues?sayforexample,youneedtondthenumberofpixels
img
=
cv2
.
imread(

home.jpg

,
0
)
hist
=
cv2
.
calcHist([img],[
0
],
None
,[
256
],[
0
,
256
])
histisa256x1array,eachvaluecorrespondstonumberofpixelsinthatimagewithitscorrespondingpixelvalue.
2.HistogramCalculationinNumpy
Numpyalsoprovidesyouafunction,
np.histogram()
.SoinsteadofcalcHist()function,youcantrybelowline:
hist,bins
=
np
.
histogram(img
.
ravel(),
256
,[
0
,
256
])
histissameaswecalculatedbefore.Butbinswillhave257elements,becauseNumpycalculatesbinsas0-0.99,
1.4.ImageProcessinginOpenCV109
OpenCV-PythonTutorialsDocumentation,Release1
Seealso:
Numpyhasanotherfunction,
np.bincount()
whichismuchfasterthan(around10X)np.histogram().Soforone-
Note:
OpenCVfunctionismorefasterthan(around40X)thannp.histogram().SostickwithOpenCVfunction.
Nowweshouldplothistograms,buthow?
PlottingHistograms
Therearetwowaysforthis,
1.
ShortWay:useMatplotlibplottingfunctions
2.
LongWay:useOpenCVdrawingfunctions
1.UsingMatplotlib
Matplotlibcomeswithahistogramplottingfunction:matplotlib.pyplot.hist()
Itdirectlyndsthehistogramandplotit.YouneednotusecalcHist()ornp.histogram()functiontondthehistogram.
Seethecodebelow:
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

home.jpg

,
0
)
plt
.
hist(img
.
ravel(),
256
,[
0
,
256
]);plt
.
show()
110Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Oryoucanusenormalplotofmatplotlib,whichwouldbegoodforBGRplot.Forthat,youneedtondthehistogram
datarst.Trybelowcode:
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

home.jpg

)
color
=
(

b

,

g

,

r

)
for
i,col
in
enumerate
(color):
histr
=
cv2
.
calcHist([img],[i],
None
,[
256
],[
0
,
256
])
plt
.
plot(histr,color
=
col)
plt
.
xlim([
0
,
256
])
plt
.
show()
Result:
Youcandeductfromtheabovegraphthat,bluehassomehighvalueareasintheimage(obviouslyitshouldbedueto
thesky)
2.UsingOpenCV
Well,hereyouadjustthevaluesofhistogramsalongwithitsbinvaluestolooklikex,ycoordinatessothatyoucan
drawitusingcv2.line()orcv2.polyline()functiontogeneratesameimageasabove.Thisisalreadyavailablewith
OpenCV-Python2ofcialsamples.
ChecktheCode
1.4.ImageProcessinginOpenCV111
OpenCV-PythonTutorialsDocumentation,Release1
ApplicationofMask
Weusedcv2.calcHist()tondthehistogramofthefullimage.Whatifyouwanttondhistogramsofsomeregions
ofanimage?Justcreateamaskimagewithwhitecolorontheregionyouwanttondhistogramandblackotherwise.
Thenpassthisasthemask.
img
=
cv2
.
imread(

home.jpg

,
0
)
#createamask
mask
=
np
.
zeros(img
.
shape[:
2
],np
.
uint8)
mask[
100
:
300
,
100
:
400
]
=
255
masked_img
=
cv2
.
bitwise_and(img,img,mask
=
mask)
#Calculatehistogramwithmaskandwithoutmask
#Checkthirdargumentformask
hist_full
=
cv2
.
calcHist([img],[
0
],
None
,[
256
],[
0
,
256
])
hist_mask
=
cv2
.
calcHist([img],[
0
],mask,[
256
],[
0
,
256
])
plt
.
subplot(
221
),plt
.
imshow(img,

gray

)
plt
.
subplot(
222
),plt
.
imshow(mask,

gray

)
plt
.
subplot(
223
),plt
.
imshow(masked_img,

gray

)
plt
.
subplot(
224
),plt
.
plot(hist_full),plt
.
plot(hist_mask)
plt
.
xlim([
0
,
256
])
plt
.
show()
Seetheresult.Inthehistogramplot,bluelineshowshistogramoffullimagewhilegreenlineshowshistogramof
maskedregion.
AdditionalResources
1.
CambridgeinColorwebsite
112Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Exercises
Histograms-2:HistogramEqualization
Goal
Inthissection,

Wewilllearntheconceptsofhistogramequalizationanduseittoimprovethecontrastofourimages.
Theory
Consideranimagewhosepixelvaluesareconnedtosomespecicrangeofvaluesonly.Foreg,brighterimagewill
haveallpixelsconnedtohighvalues.Butagoodimagewillhavepixelsfromallregionsoftheimage.Soyou
Iwouldrecommendyoutoreadthewikipediapageon
HistogramEqualization
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

wiki.jpg

,
0
)
hist,bins
=
np
.
histogram(img
.
flatten(),
256
,[
0
,
256
])
cdf
=
hist
.
cumsum()
cdf_normalized
=
cdf
*
hist
.
max()
/
cdf
.
max()
plt
.
plot(cdf_normalized,color
=

b

)
plt
.
hist(img
.
flatten(),
256
,[
0
,
256
],color
=

r

)
plt
.
xlim([
0
,
256
])
plt
.
legend((

cdf

,

histogram

),loc
=

upperleft

)
plt
.
show()
1.4.ImageProcessinginOpenCV113
OpenCV-PythonTutorialsDocumentation,Release1
Youcanseehistogramliesinbrighterregion.Weneedthefullspectrum.Forthat,weneedatransformationfunction
whichmapstheinputpixelsinbrighterregiontooutputpixelsinfullregion.Thatiswhathistogramequalizationdoes.
Nowwendtheminimumhistogramvalue(excluding0)andapplythehistogramequalizationequationasgivenin
wikipage.ButIhaveusedhere,themaskedarrayconceptarrayfromNumpy.Formaskedarray,alloperationsare
performedonnon-maskedelements.YoucanreadmoreaboutitfromNumpydocsonmaskedarrays.
cdf_m
=
np
.
ma
.
masked_equal(cdf,
0
)
cdf_m
=
(cdf_m
-
cdf_m
.
min())
*
255
/
(cdf_m
.
max()
-
cdf_m
.
min())
cdf
=
np
.
ma
.
filled(cdf_m,
0
)
.
astype(

uint8

)
Nowwehavethelook-uptablethatgivesustheinformationonwhatistheoutputpixelvalueforeveryinputpixel
value.Sowejustapplythetransform.
img2
=
cdf[img]
Nowwecalculateitshistogramandcdfasbefore(youdoit)andresultlookslikebelow:
114Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Anotherimportantfeatureisthat,eveniftheimagewasadarkerimage(insteadofabrighteroneweused),after
img
=
cv2
.
imread(

wiki.jpg

,
0
)
equ
=
cv2
.
equalizeHist(img)
res
=
np
.
hstack((img,equ))
#stackingimagesside-by-side
cv2
.
imwrite(

res.png

,res)
1.4.ImageProcessinginOpenCV115
OpenCV-PythonTutorialsDocumentation,Release1
Sonowyoucantakedifferentimageswithdifferentlightconditions,equalizeitandchecktheresults.
Histogramequalizationisgoodwhenhistogramoftheimageisconnedtoaparticularregion.Itwon'tworkgoodin
placeswherethereislargeintensityvariationswherehistogramcoversalargeregion,iebothbrightanddarkpixels
arepresent.PleasechecktheSOFlinksinAdditionalResources.
CLAHE(ContrastLimitedAdaptiveHistogramEqualization)
Thersthistogramequalizationwejustsaw,considerstheglobalcontrastoftheimage.Inmanycases,itisnotagood
idea.Forexample,belowimageshowsaninputimageanditsresultafterglobalhistogramequalization.
116Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
1.4.ImageProcessinginOpenCV117
OpenCV-PythonTutorialsDocumentation,Release1
Itistruethatthebackgroundcontrasthasimprovedafterhistogramequalization.Butcomparethefaceofstatuein
bothimages.Welostmostoftheinformationthereduetoover-brightness.Itisbecauseitshistogramisnotconned
import
numpy
as
np
import
cv2
img
=
cv2
.
imread(

tsukuba_l.png

,
0
)
#createaCLAHEobject(Argumentsareoptional).
clahe
=
cv2
.
createCLAHE(clipLimit
=
2.0
,tileGridSize
=
(
8
,
8
))
cl1
=
clahe
.
apply(img)
cv2
.
imwrite(

clahe_2.jpg

,cl1)
Seetheresultbelowandcompareitwithresultsabove,especiallythestatueregion:
AdditionalResources
1.
Wikipediapageon
HistogramEqualization
2.
MaskedArraysinNumpy
118Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AlsochecktheseSOFquestionsregardingcontrastadjustment:
3.
HowcanIadjustcontrastinOpenCVinC?
4.
HowdoIequalizecontrast&brightnessofimagesusingopencv?
Exercises
Histograms-3:2DHistograms
Goal
Inthischapter,wewilllearntondandplot2Dhistograms.Itwillbehelpfulincomingchapters.
Introduction
Intherstarticle,wecalculatedandplottedone-dimensionalhistogram.Itiscalledone-dimensionalbecausewe
aretakingonlyonefeatureintoourconsideration,iegrayscaleintensityvalueofthepixel.Butintwo-dimensional
histograms,youconsidertwofeatures.NormallyitisusedforndingcolorhistogramswheretwofeaturesareHue&
Saturationvaluesofeverypixel.
Thereisa
pythonsampleintheofcialsamples
alreadyforndingcolorhistograms.Wewilltrytounderstandhow
tocreatesuchacolorhistogram,anditwillbeusefulinunderstandingfurthertopicslikeHistogramBack-Projection.
2DHistograminOpenCV
Itisquitesimpleandcalculatedusingthesamefunction,
cv2.calcHist()
.Forcolorhistograms,weneedtoconvertthe
imagefromBGRtoHSV.(Remember,for1Dhistogram,weconvertedfromBGRtoGrayscale).For2Dhistograms,
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

home.jpg

)
hsv
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2HSV)
hist
=
cv2
.
calcHist([hsv],[
0
,
1
],
None
,[
180
,
256
],[
0
,
180
,
0
,
256
])
That'sit.
2DHistograminNumpy
Numpyalsoprovidesaspecicfunctionforthis:
np.histogram2d()
.(Remember,for1Dhistogramweused
np.histogram()
).
1.4.ImageProcessinginOpenCV119
OpenCV-PythonTutorialsDocumentation,Release1
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

home.jpg

)
hsv
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2HSV)
hist,xbins,ybins
=
np
.
histogram2d(h
.
ravel(),s
.
ravel(),[
180
,
256
],[[
0
,
180
],[
0
,
256
]])
FirstargumentisHplane,secondoneistheSplane,thirdisnumberofbinsforeachandfourthistheirrange.
Nowwecancheckhowtoplotthiscolorhistogram.
Plotting2DHistograms
Note:
Whileusingthisfunction,remember,interpolationagshouldbe
nearest
Considercode:
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

home.jpg

)
hsv
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2HSV)
hist
=
cv2
.
calcHist([hsv],[
0
,
1
],
None
,[
180
,
256
],[
0
,
180
,
0
,
256
])
plt
.
imshow(hist,interpolation
=

nearest

)
plt
.
show()
Belowistheinputimageanditscolorhistogramplot.XaxisshowsSvaluesandYaxisshowsHue.
120Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Inhistogram,youcanseesomehighvaluesnearH=100andS=200.Itcorrespondstoblueofsky.Similarlyanother
peakcanbeseennearH=25andS=100.Itcorrespondstoyellowofthepalace.Youcanverifyitwithanyimage
editingtoolslikeGIMP.
Youcanclearlyseeinthehistogramwhatcolorsarepresent,blueisthere,yellowisthere,andsomewhitedueto
chessboardisthere.Nice!!!
AdditionalResources
Exercises
Histogram-4:HistogramBackprojection
1.4.ImageProcessinginOpenCV121
OpenCV-PythonTutorialsDocumentation,Release1
Goal
Inthischapter,wewilllearnabouthistogrambackprojection.
Theory
Itwasproposedby
MichaelJ.Swain,DanaH.Ballard
intheirpaper
Indexingviacolorhistograms
.
Whatisitactuallyinsimplewords?
Itisusedforimagesegmentationorndingobjectsofinterestinanimage.In
simplewords,itcreatesanimageofthesamesize(butsinglechannel)asthatofourinputimage,whereeachpixel
correspondstotheprobabilityofthatpixelbelongingtoourobject.Inmoresimplerworlds,theoutputimagewill
haveourobjectofinterestinmorewhitecomparedtoremainingpart.Well,thatisanintuitiveexplanation.(Ican't
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
#roiistheobjectorregionofobjectweneedtofind
roi
=
cv2
.
imread(

rose_red.png

)
hsv
=
cv2
.
cvtColor(roi,cv2
.
COLOR_BGR2HSV)
2.Findtheratio
R
=
M
I
h,s,v
=
cv2
.
split(hsvt)
B
=
R[h
.
ravel(),s
.
ravel()]
B
=
np
.
minimum(B,
1
)
B
=
B
.
reshape(hsvt
.
shape[:
2
])
3.Nowapplyaconvolutionwithacirculardisc,
B
=
D

B
,whereDisthedisckernel.
disc
=
cv2
.
122Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
B
=
np
.
uint8(B)
cv2
.
normalize(B,B,
0
,
255
,cv2
.
NORM_MINMAX)
4.Nowthelocationofmaximumintensitygivesusthelocationofobject.Ifweareexpectingaregionintheimage,
thresholdingforasuitablevaluegivesaniceresult.
That'sit!!
BackprojectioninOpenCV
OpenCVprovidesaninbuiltfunction
cv2.calcBackProject()
import
cv2
import
numpy
as
np
roi
=
cv2
.
imread(

rose_red.png

)
hsv
=
cv2
.
cvtColor(roi,cv2
.
COLOR_BGR2HSV)
BelowisoneexampleIworkedwith.IusedtheregioninsidebluerectangleassampleobjectandIwantedtoextract
thefullground.
1.4.ImageProcessinginOpenCV123
OpenCV-PythonTutorialsDocumentation,Release1
124Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
“Indexingviacolorhistograms”,Swain,MichaelJ.,Thirdinternationalconferenceoncomputervision,1990.
Exercises
ImageTransformsinOpenCV

FourierTransform
LearntondtheFourierTransformofimages
1.4.ImageProcessinginOpenCV125
OpenCV-PythonTutorialsDocumentation,Release1
FourierTransform
Goal
Inthissection,wewilllearn

TondtheFourierTransformofimagesusingOpenCV

ToutilizetheFFTfunctionsavailableinNumpy

SomeapplicationsofFourierTransform

Wewillseefollowingfunctions:
cv2.dft()
,
cv2.idft()
2
inboththedirections.Thisissimplydonebythefunction,
np.fft.fftshift()
.(Itismoreeasiertoanalyze).Onceyoufoundthefrequencytransform,youcanndthemagnitude
spectrum.
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

messi5.jpg

,
0
)
f
=
np
.
fft
.
fft2(img)
fshift
=
np
.
fft
.
fftshift(f)
magnitude_spectrum
=
20
*
np
.
log(np
.
abs(fshift))
126Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
plt
.
subplot(
121
),plt
.
imshow(img,cmap
=

gray

)
plt
.
title(

InputImage

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
122
),plt
.
imshow(magnitude_spectrum,cmap
=

gray

)
plt
.
title(

MagnitudeSpectrum

),plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Resultlooklikebelow:
See,Youcanseemorewhiterregionatthecentershowinglowfrequencycontentismore.
SoyoufoundthefrequencytransformNowyoucandosomeoperationsinfrequencydomain,likehighpassltering
andreconstructtheimage,iendinverseDFT.Forthatyousimplyremovethelowfrequenciesbymaskingwitha
rectangularwindowofsize60x60.Thenapplytheinverseshiftusing
np.fft.ifftshift()
sothatDCcomponentagain
comeatthetop-leftcorner.ThenndinverseFFTusing
np.ifft2()
function.Theresult,again,willbeacomplex
number.Youcantakeitsabsolutevalue.
rows,cols
=
img
.
shape
crow,ccol
=
rows
/
2
,cols
/
2
fshift[crow
-
30
:crow
+
30
,ccol
-
30
:ccol
+
30
]
=
0
f_ishift
=
np
.
fft
.
ifftshift(fshift)
img_back
=
np
.
fft
.
ifft2(f_ishift)
img_back
=
np
.
abs(img_back)
plt
.
subplot(
131
),plt
.
imshow(img,cmap
=

gray

)
plt
.
title(

InputImage

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
132
),plt
.
imshow(img_back,cmap
=

gray

)
plt
.
title(

ImageafterHPF

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
133
),plt
.
imshow(img_back)
plt
.
title(

Resultlooklikebelow:
1.4.ImageProcessinginOpenCV127
OpenCV-PythonTutorialsDocumentation,Release1
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

messi5.jpg

,
0
)
dft
=
cv2
.
dft(np
.
float32(img),flags
=
cv2
.
DFT_COMPLEX_OUTPUT)
dft_shift
=
np
.
fft
.
fftshift(dft)
magnitude_spectrum
=
20
*
np
.
log(cv2
.
magnitude(dft_shift[:,:,
0
],dft_shift[:,:,
1
]))
plt
.
subplot(
121
),plt
.
imshow(img,cmap
=

gray

)
plt
.
title(

InputImage

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
122
),plt
.
imshow(magnitude_spectrum,cmap
=

gray

)
plt
.
title(

MagnitudeSpectrum

),plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Note:
Youcanalsouse
cv2.cartToPolar()
So,nowwehavetodoinverseDFT.Inprevioussession,wecreatedaHPF,thistimewewillseehowtoremovehigh
frequencycontentsintheimage,ieweapplyLPFtoimage.Itactuallyblurstheimage.Forthis,wecreateamaskrst
withhighvalue(1)atlowfrequencies,iewepasstheLFcontent,and0atHFregion.
rows,cols
=
img
.
shape
crow,ccol
=
rows
/
2
,cols
/
2
128Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
#createamaskfirst,centersquareis1,remainingallzeros
mask
=
np
.
zeros((rows,cols,
2
),np
.
uint8)
mask[crow
-
30
:crow
+
30
,ccol
-
30
:ccol
+
30
]
=
1
#applymaskandinverseDFT
fshift
=
dft_shift
*
mask
f_ishift
=
np
.
fft
.
ifftshift(fshift)
img_back
=
cv2
.
idft(f_ishift)
img_back
=
cv2
.
magnitude(img_back[:,:,
0
],img_back[:,:,
1
])
plt
.
subplot(
121
),plt
.
imshow(img,cmap
=

gray

)
plt
.
title(

InputImage

),plt
.
xticks([]),plt
.
yticks([])
plt
.
subplot(
122
),plt
.
imshow(img_back,cmap
=

gray

)
plt
.
title(

MagnitudeSpectrum

),plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Seetheresult:
Note:
Asusual,OpenCVfunctions
cv2.dft()
and
cv2.idft()
arefasterthanNumpycounterparts.ButNumpyfunctions
PerformanceOptimizationofDFT
In[
16
]:img
=
cv2
.
imread(

messi5.jpg

,
0
)
In[
17
]:rows,cols
=
img
.
shape
In[
18
]:
print
rows,cols
342
548
In[
19
]:nrows
=
cv2
.
1.4.ImageProcessinginOpenCV129
OpenCV-PythonTutorialsDocumentation,Release1
nimg
=
np
.
zeros((nrows,ncols))
nimg[:rows,:cols]
=
img
OR:
right
=
ncols
-
cols
bottom
=
nrows
-
rows
bordertype
=
cv2
.
BORDER_CONSTANT
#justtoavoidlinebreakupinPDFfile
nimg
=
cv2
.
copyMakeBorder(img,
0
,bottom,
0
,right,bordertype,value
=
0
)
NowwecalculatetheDFTperformancecomparisonofNumpyfunction:
In[
22
]:
%
timeitfft1
=
np
.
fft
.
fft2(img)
10
loops,bestof
3
:
40.9
msperloop
In[
23
]:
%
timeitfft2
=
np
.
fft
.
fft2(img,[nrows,ncols])
100
loops,bestof
3
:
10.4
msperloop
Itshowsa4xspeedup.NowwewilltrythesamewithOpenCVfunctions.
In[
24
]:
%
timeitdft1
=
cv2
.
dft(np
.
float32(img),flags
=
cv2
.
DFT_COMPLEX_OUTPUT)
100
loops,bestof
3
:
13.5
msperloop
In[
27
]:
%
timeitdft2
=
cv2
.
dft(np
.
float32(nimg),flags
=
cv2
.
DFT_COMPLEX_OUTPUT)
100
loops,bestof
3
:
3.11
msperloop
Italsoshowsa4xspeed-up.YoucanalsoseethatOpenCVfunctionsarearound3xfasterthanNumpyfunctions.This
canbetestedforinverseFFTalso,andthatisleftasanexerciseforyou.
WhyLaplacianisaHighPassFilter?
Asimilarquestionwasaskedinaforum.Thequestionis,whyLaplacianisahighpasslter?WhySobelisaHPF?
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
130Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
#sobelinydirection
sobel_y
=
np
.
array([[
-
1
,
-
2
,
-
1
],
[
0
,
0
,
0
],
[
1
,
2
,
1
]])
#laplacian
laplacian
=
np
.
array([[
0
,
1
,
0
],
[
1
,
-
4
,
1
],
[
0
,
1
,
0
]])
filters
=
[mean_filter,gaussian,laplacian,sobel_x,sobel_y,scharr]
filter_name
=
[

mean_filter

,

gaussian

,

laplacian

,

sobel_x

,\

sobel_y

,

scharr_x

]
fft_filters
=
[np
.
fft
.
fft2(x)
for
x
in
filters]
fft_shift
=
[np
.
fft
.
fftshift(y)
for
y
in
fft_filters]
mag_spectrum
=
[np
.
log(np
.
abs(z)
+
1
)
for
z
in
fft_shift]
for
i
in
xrange(
6
):
plt
.
subplot(
2
,
3
,i
+
1
),plt
.
imshow(mag_spectrum[i],cmap
=

gray

)
plt
.
title(filter_name[i]),plt
.
xticks([]),plt
.
yticks([])
plt
.
show()
Seetheresult:
Fromimage,youcanseewhatfrequencyregioneachkernelblocks,andwhatregionitpasses.Fromthatinformation,
wecansaywhyeachkernelisaHPForaLPF
1.4.ImageProcessinginOpenCV131
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
AnIntuitiveExplanationofFourierTheory
byStevenLehar
2.
FourierTransform
atHIPR
3.
Whatdoesfrequencydomaindenoteincaseofimages?
Exercises
TemplateMatching
Goals
Inthischapter,youwilllearn

TondobjectsinanimageusingTemplateMatching

Youwillseethesefunctions:
cv2.matchTemplate()
,
cv2.minMaxLoc()
Theory
Note:
Ifyouareusing
cv2.TM_SQDIFF
TemplateMatchinginOpenCV
Here,asanexample,wewillsearchforMessi'sfaceinhisphoto.SoIcreatedatemplateasbelow:
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

messi5.jpg

,
0
)
img2
=
img
.
copy()
template
=
cv2
.
imread(

template.jpg

,
0
)
132Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
w,h
=
template
.
shape[::
-
1
]
Seetheresultsbelow:

cv2.TM_CCOEFF

cv2.TM_CCOEFF_NORMED
1.4.ImageProcessinginOpenCV133
OpenCV-PythonTutorialsDocumentation,Release1

cv2.TM_CCORR

cv2.TM_CCORR_NORMED

cv2.TM_SQDIFF
134Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1

cv2.TM_SQDIFF_NORMED
Youcanseethattheresultusing
cv2.TM_CCORR
isnotgoodasweexpected.
TemplateMatchingwithMultipleObjects
Intheprevioussection,wesearchedimageforMessi'sface,whichoccursonlyonceintheimage.Supposeyouare
searchingforanobjectwhichhasmultipleoccurances,
cv2.minMaxLoc()
won'tgiveyouallthelocations.Inthat
case,wewillusethresholding.Sointhisexample,wewilluseascreenshotofthefamousgame
Mario
andwewill
ndthecoinsinit.
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img_rgb
=
cv2
.
imread(

mario.png

)
img_gray
=
cv2
.
cvtColor(img_rgb,cv2
.
COLOR_BGR2GRAY)
template
=
cv2
.
imread(

mario_coin.png

,
0
)
w,h
=
template
.
shape[::
-
1
]
res
=
cv2
.
matchTemplate(img_gray,template,cv2
.
TM_CCOEFF_NORMED)
threshold
=
0.8
1.4.ImageProcessinginOpenCV135
OpenCV-PythonTutorialsDocumentation,Release1
loc
=
np
.
where(res

=
threshold)
for
pt
in
zip
(
*
loc[::
-
1
]):
cv2
.
rectangle(img_rgb,pt,(pt[
0
]
+
w,pt[
1
]
+
h),(
0
,
0
,
255
),
2
)
cv2
.
imwrite(

res.png

,img_rgb)
Result:
AdditionalResources
Exercises
HoughLineTransform
Goal
Inthischapter,

WewillunderstandtheconceptofHoughTranform.

136Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Soiflineispassingbelowtheorigin,itwillhaveapositiverhoandanglelessthan180.Ifitisgoingabovetheorigin,
insteadoftakinganglegreaterthan180,angleistakenlessthan180,andrhoistakennegative.Anyverticallinewill
have0degreeandhorizontallineswillhave90degree.
HoughTranforminOpenCV
EverythingexplainedaboveisencapsulatedintheOpenCVfunction,
cv2.HoughLines()
import
cv2
import
numpy
as
np
1.4.ImageProcessinginOpenCV137
OpenCV-PythonTutorialsDocumentation,Release1
img
=
cv2
.
imread(

dave.jpg

)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
edges
=
cv2
.
Canny(gray,
50
,
150
,apertureSize
=
3
)
lines
=
cv2
.
HoughLines(edges,
1
,np
.
pi
/
180
,
200
)
for
Checktheresultsbelow:
138Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
ProbabilisticHoughTransform
Inthehoughtransform,youcanseethatevenforalinewithtwoarguments,ittakesalotofcomputation.Probabilistic
HoughTransformisanoptimizationofHoughTransformwesaw.Itdoesn'ttakeallthepointsintoconsideration,
1.4.ImageProcessinginOpenCV139
OpenCV-PythonTutorialsDocumentation,Release1
140Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

dave.jpg

)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
edges
=
cv2
.
Canny(gray,
50
,
150
,apertureSize
=
3
)
minLineLength
=
100
maxLineGap
=
10
lines
=
cv2
.
HoughLinesP(edges,
1
,np
.
pi
/
180
,
100
,minLineLength,maxLineGap)
for
x1,y1,x2,y2
in
lines[
0
]:
cv2
.
line(img,(x1,y1),(x2,y2),(
0
,
255
,
0
),
2
)
cv2
.
imwrite(

houghlines5.jpg

,img)
Seetheresultsbelow:
1.4.ImageProcessinginOpenCV141
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
HoughTransformonWikipedia
Exercises
HoughCircleTransform
Goal
Inthischapter,

WewilllearntouseHoughTransformtondcirclesinanimage.

Wewillseethesefunctions:
cv2.HoughCircles()
Theory
Acircleisrepresentedmathematicallyas
(
x

x
center
)
2
+(
y

y
center
)
2
=
r
2
where
(
x
center
;y
center
)
isthecenter
ofthecircle,and
r
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

opencv_logo.png

,
0
)
img
=
cv2
.
medianBlur(img,
5
)
cimg
=
cv2
.
cvtColor(img,cv2
.
COLOR_GRAY2BGR)
circles
=
cv2
.
HoughCircles(img,cv2
.
HOUGH_GRADIENT,
1
,
20
,
param1
=
50
,param2
=
30
,minRadius
=
0
,maxRadius
=
0
)
circles
=
np
.
uint16(np
.
around(circles))
for
i
in
circles[
0
,:]:
#drawtheoutercircle
cv2
.
circle(cimg,(i[
0
],i[
1
]),i[
2
],(
0
,
255
,
0
),
2
)
#drawthecenterofthecircle
cv2
.
circle(cimg,(i[
0
],i[
1
]),
2
,(
0
,
0
,
255
),
3
)
cv2
.
imshow(

Resultisshownbelow:
142Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
ImageSegmentationwithWatershedAlgorithm
Goal
Inthischapter,

Wewilllearntousemarker-basedimagesegmentationusingwatershedalgorithm

Wewillsee:
cv2.watershed()
Theory
Anygrayscaleimagecanbeviewedasatopographicsurfacewherehighintensitydenotespeaksandhillswhilelow
intensitydenotesvalleys.Youstartllingeveryisolatedvalleys(localminima)withdifferentcoloredwater(labels).
Asthewaterrises,dependingonthepeaks(gradients)nearby,waterfromdifferentvalleys,obviouslywithdifferent
colorswillstarttomerge.Toavoidthat,youbuildbarriersinthelocationswherewatermerges.Youcontinuethework
ofllingwaterandbuildingbarriersuntilallthepeaksareunderwater.Thenthebarriersyoucreatedgivesyouthe
segmentationresult.Thisisthe“philosophy”behindthewatershed.Youcanvisitthe
CMMwebpageonwatershed
to
understanditwiththehelpofsomeanimations.
Butthisapproachgivesyouoversegmentedresultduetonoiseoranyotherirregularitiesintheimage.SoOpenCV
implementedamarker-basedwatershedalgorithmwhereyouspecifywhichareallvalleypointsaretobemergedand
whicharenot.Itisaninteractiveimagesegmentation.Whatwedoistogivedifferentlabelsforourobjectweknow.
Labeltheregionwhichwearesureofbeingtheforegroundorobjectwithonecolor(orintensity),labeltheregion
whichwearesureofbeingbackgroundornon-objectwithanothercolorandnallytheregionwhichwearenotsure
ofanything,labelitwith0.Thatisourmarker.Thenapplywatershedalgorithm.Thenourmarkerwillbeupdated
withthelabelswegave,andtheboundariesofobjectswillhaveavalueof-1.
1.4.ImageProcessinginOpenCV143
OpenCV-PythonTutorialsDocumentation,Release1
Code
BelowwewillseeanexampleonhowtousetheDistanceTransformalongwithwatershedtosegmentmutually
touchingobjects.
Considerthecoinsimagebelow,thecoinsaretouchingeachother.Evenifyouthresholdit,itwillbetouchingeach
other.
Westartwithndinganapproximateestimateofthecoins.Forthat,wecanusetheOtsu'sbinarization.
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

coins.png

)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
Result:
144Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Nowweneedtoremoveanysmallwhitenoisesintheimage.Forthatwecanusemorphologicalopening.Toremove
anysmallholesintheobject,wecanusemorphologicalclosing.So,nowweknowforsurethatregionneartocenter
ofobjectsareforegroundandregionmuchawayfromtheobjectarebackground.Onlyregionwearenotsureisthe
boundaryregionofcoins.
Soweneedtoextracttheareawhichwearesuretheyarecoins.Erosionremovestheboundarypixels.Sowhatever
remaining,wecanbesureitiscoin.Thatwouldworkifobjectswerenottouchingeachother.Butsincetheyare
touchingeachother,anothergoodoptionwouldbetondthedistancetransformandapplyaproperthreshold.Next
weneedtondtheareawhichwearesuretheyarenotcoins.Forthat,wedilatetheresult.Dilationincreasesobject
boundarytobackground.Thisway,wecanmakesurewhateverregioninbackgroundinresultisreallyabackground,
sinceboundaryregionisremoved.Seetheimagebelow.
1.4.ImageProcessinginOpenCV145
OpenCV-PythonTutorialsDocumentation,Release1
#noiseremoval
kernel
=
np
.
ones((
3
,
3
),np
.
uint8)
opening
=
cv2
.
morphologyEx(thresh,cv2
.
MORPH_OPEN,kernel,iterations
=
2
)
#surebackgroundarea
sure_bg
=
cv2
.
dilate(opening,kernel,iterations
=
3
)
#Findingsureforegroundarea
dist_transform
=
cv2
.
146Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Nowweknowforsurewhichareregionofcoins,whicharebackgroundandall.Sowecreatemarker(itisanarrayof
samesizeasthatoforiginalimage,butwithint32datatype)andlabeltheregionsinsideit.Theregionsweknowfor
#Markerlabelling
1.4.ImageProcessinginOpenCV147
OpenCV-PythonTutorialsDocumentation,Release1
Nowourmarkerisready.Itistimefornalstep,applywatershed.Thenmarkerimagewillbemodied.Theboundary
regionwillbemarkedwith-1.
markers
=
cv2
.
watershed(img,markers)
img[markers
==
-
1
]
=
[
255
,
0
,
0
]
Seetheresultbelow.Forsomecoins,theregionwheretheytoucharesegmentedproperlyandforsome,theyarenot.
148Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
CMMpageon
WatershedTranformation
Exercises
1.
OpenCVsampleshasaninteractivesampleonwatershedsegmentation,
watershed.py
.Runit,Enjoyit,then
learnit.
InteractiveForegroundExtractionusingGrabCutAlgorithm
Goal
Inthischapter

WewillseeGrabCutalgorithmtoextractforegroundinimages

Wewillcreateaninteractiveapplicationforthis.
Theory
GrabCutalgorithmwasdesignedbyCarstenRother,VladimirKolmogorov&AndrewBlakefromMicrosoftResearch
Cambridge,UK.intheirpaper,
“GrabCut”:interactiveforegroundextractionusingiteratedgraphcuts
.Analgorithm
wasneededforforegroundextractionwithminimaluserinteraction,andtheresultwasGrabCut.
Howitworksfromuserpointofview?Initiallyuserdrawsarectanglearoundtheforegroundregion(foreground
1.4.ImageProcessinginOpenCV149
OpenCV-PythonTutorialsDocumentation,Release1
andviceversa.Inthatcase,userneedtodonetouch-ups.Justgivesomestrokesontheimageswheresomefaulty
resultsarethere.Strokesbasicallysays
“Hey,thisregionshouldbeforeground,youmarkeditbackground,correctit
innextiteration”
Sowhathappensinbackground?

Userinputstherectangle.Everythingoutsidethisrectanglewillbetakenassurebackground(Thatisthe
reasonitismentionedbeforethatyourrectangleshouldincludealltheobjects).Everythinginsiderectangle
isunknown.Similarlyanyuserinputspecifyingforegroundandbackgroundareconsideredashard-labelling
whichmeanstheywon'tchangeintheprocess.

Computerdoesaninitiallabellingdepedingonthedatawegave.Itlabelstheforegroundandbackgroundpixels
(orithard-labels)

NowaGaussianMixtureModel(GMM)isusedtomodeltheforegroundandbackground.

Dependingonthedatawegave,GMMlearnsandcreatenewpixeldistribution.Thatis,theunknownpixels
arelabelledeitherprobableforegroundorprobablebackgrounddependingonitsrelationwiththeotherhard-
labelledpixelsintermsofcolorstatistics(Itisjustlikeclustering).

Agraphisbuiltfromthispixeldistribution.Nodesinthegraphsarepixels.Additionaltwonodesareadded,
Sourcenode
and
Sinknode
.EveryforegroundpixelisconnectedtoSourcenodeandeverybackgroundpixel
isconnectedtoSinknode.

Theweightsofedgesconnectingpixelstosourcenode/endnodearedenedbytheprobabilityofapixelbeing
150Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Demo
NowwegoforgrabcutalgorithmwithOpenCV.OpenCVhasthefunction,
cv2.grabCut()
forthis.Wewillseeits
argumentsrst:

img
-Inputimage

mask
-Itisamaskimagewherewespecifywhichareasarebackground,foregroundorprobableback-
1.4.ImageProcessinginOpenCV151
OpenCV-PythonTutorialsDocumentation,Release1
wemodifythemasksuchthatall0-pixelsand2-pixelsareputto0(iebackground)andall1-pixelsand3-pixelsare
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

messi5.jpg

)
mask
=
np
.
zeros(img
.
shape[:
2
],np
.
uint8)
bgdModel
=
np
.
zeros((
1
,
65
),np
.
float64)
fgdModel
=
np
.
zeros((
1
,
65
),np
.
float64)
rect
=
(
50
,
50
,
450
,
290
)
cv2
.
grabCut(img,mask,rect,bgdModel,fgdModel,
5
,cv2
.
GC_INIT_WITH_RECT)
mask2
=
np
.
where((mask
==
2
)
|
(mask
==
0
),
0
,
1
)
.
astype(

uint8

)
img
=
img
*
mask2[:,:,np
.
newaxis]
plt
.
imshow(img),plt
.
colorbar(),plt
.
show()
Seetheresultsbelow:
Oops,Messi'shairisgone.
WholikesMessiwithouthishair?
Weneedtobringitback.Sowewillgivethereane
touchupwith1-pixel(sureforeground).Atthesametime,Somepartofgroundhascometopicturewhichwedon't
want,andalsosomelogo.Weneedtoremovethem.Therewegivesome0-pixeltouchup(surebackground).Sowe
modifyourresultingmaskinpreviouscaseaswetoldnow.
WhatIactuallydidisthat,Iopenedinputimageinpaintapplicationandaddedanotherlayertotheimage.Using
152Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
#newmaskisthemaskimageImanuallylabelled
newmask
=
cv2
.
imread(

newmask.png

,
0
)
#whereeveritismarkedwhite(sureforeground),changemask=1
#whereeveritismarkedblack(surebackground),changemask=0
mask[newmask
==
0
]
=
0
mask[newmask
==
255
]
=
1
mask,bgdModel,fgdModel
=
cv2
.
grabCut(img,mask,
None
,bgdModel,fgdModel,
5
,cv2
.
GC_INIT_
,
!
WITH_MASK)
mask
=
np
.
where((mask
==
2
)
|
(mask
==
0
),
0
,
1
)
.
astype(

uint8

)
img
=
img
*
mask[:,:,np
.
newaxis]
plt
.
imshow(img),plt
.
colorbar(),plt
.
show()
Seetheresultbelow:
Sothat'sit.Hereinsteadofinitializinginrectmode,youcandirectlygointomaskmode.Justmarktherectangle
areainmaskimagewith2-pixelor3-pixel(probablebackground/foreground).Thenmarkoursure_foregroundwith
1-pixelaswedidinsecondexample.ThendirectlyapplythegrabCutfunctionwithmaskmode.
AdditionalResources
Exercises
1.
OpenCVsamplescontainasample
grabcut.py
whichisaninteractivetoolusinggrabcut.Checkit.Also
watchthis
youtubevideo
onhowtouseit.
2.
Here,youcanmakethisintoainteractivesamplewithdrawingrectangleandstrokeswithmouse,createtrackbar
OpenCV-PythonTutorialsDocumentation,Release1
Whatarethemainfeaturesinanimage?Howcanndingthosefeaturesbe
usefultous?

Okay,Cornersaregoodfeatures?Buthowdowendthem?


IntroductiontoSIFT(Scale-InvariantFeatureTransform)

IntroductiontoSURF(Speeded-UpRobustFeatures)
SIFTisreallygood,butnotfastenough,sopeoplecameupwithaspeeded-
upversioncalledSURF.

154Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1

BRIEF(BinaryRobustIndependentElementaryFeatures)
SIFTusesafeaturedescriptorwith128oatingpointnumbers.Consider
thousandsofsuchfeatures.Ittakeslotsofmemoryandmoretimefor
matching.Wecancompressittomakeitfaster.Butstillwehavetocal-
culateitrst.TherecomesBRIEFwhichgivestheshortcuttondbinary
descriptorswithlessmemory,fastermatching,stillhigherrecognitionrate.

ORB(OrientedFASTandRotatedBRIEF)
SIFTandSURFaregoodinwhattheydo,butwhatifyouhavetopaya
fewdollarseveryyeartousetheminyourapplications?Yeah,theyare
patented!!!Tosolvethatproblem,OpenCVdevscameupwithanew
“FREE”alternativetoSIFT&SURF,andthatisORB.

FeatureMatching

FeatureMatching+HomographytondObjects
OpenCV-PythonTutorialsDocumentation,Release1
UnderstandingFeatures
Goal
156Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Imageisverysimple.Atthetopofimage,sixsmallimagepatchesaregiven.Questionforyouistondtheexact
locationofthesepatchesintheoriginalimage.Howmanycorrectresultsyoucannd?
AandBareatsurfaces,andtheyarespreadinalotofarea.Itisdifculttondtheexactlocationofthesepatches.
CandDaremuchmoresimpler.Theyareedgesofthebuilding.Youcanndanapproximatelocation,butexact
locationisstilldifcult.Itisbecause,alongtheedge,itissameeverywhere.Normaltotheedge,itisdifferent.So
OpenCV-PythonTutorialsDocumentation,Release1
Justlikeabove,bluepatchisatareaanddifculttondandtrack.Whereveryoumovethebluepatch,itlooksthe
same.Forblackpatch,itisanedge.Ifyoumoveitinverticaldirection(i.e.alongthegradient)itchanges.Putalong
theedge(paralleltoedge),itlooksthesame.Andforredpatch,itisacorner.Whereveryoumovethepatch,itlooks
different,meansitisunique.Sobasically,cornersareconsideredtobegoodfeaturesinanimage.(Notjustcorners,
insomecasesblobsareconsideredgoodfeatures).
Sonowweansweredourquestion,“whatarethesefeatures?”.Butnextquestionarises.Howdowendthem?Or
howdowendthecorners?.Thatalsoweansweredinanintuitiveway,i.e.,lookfortheregionsinimageswhichhave
maximumvariationwhenmoved(byasmallamount)inallregionsaroundit.Thiswouldbeprojectedintocomputer
languageincomingchapters.Sondingtheseimagefeaturesiscalled
158Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Theory
Inlastchapter,wesawthatcornersareregionsintheimagewithlargevariationinintensityinallthedirections.One
earlyattempttondthesecornerswasdoneby
ChrisHarris&MikeStephens
intheirpaper
ACombinedCorner
{z
}
windowfunction
[
I
(
x
+
u;y
+
v
)
|
{z
}
shiftedintensity

I
(
x;y
)
|
{z
}
intensity
]
2
Windowfunctioniseitherarectangularwindoworgaussianwindowwhichgivesweightstopixelsunderneath.
Wehavetomaximizethisfunction
E
(
u;v
)
OpenCV-PythonTutorialsDocumentation,Release1
import
cv2
import
numpy
as
np
filename
=

chessboard.jpg

img
=
cv2
.
imread(filename)
160Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
gray
=
np
.
float32(gray)
dst
=
cv2
.
cornerHarris(gray,
2
,
3
,
0.04
)
#resultisdilatedformarkingthecorners,notimportant
dst
=
cv2
.
dilate(dst,
None
)
#Thresholdforanoptimalvalue,itmayvarydependingontheimage.
img[dst

0.01
*
dst
.
max()]
=
[
0
,
0
,
255
]
cv2
.
imshow(

dst

,img)
if
cv2
.
waitKey(
0
)
&
0xff
==
27
:
cv2
.
destroyAllWindows()
Belowarethethreeresults:
OpenCV-PythonTutorialsDocumentation,Release1
CornerwithSubPixelAccuracy
import
cv2
import
numpy
as
np
filename
=

chessboard2.jpg

img
=
cv2
.
imread(filename)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
#findHarriscorners
gray
=
np
.
float32(gray)
dst
=
cv2
.
cornerHarris(gray,
2
,
3
,
0.04
)
dst
=
cv2
.
dilate(dst,
None
)
Belowistheresult,wheresomeimportantlocationsareshowninzoomedwindowtovisualize:
162Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
OpenCV-PythonTutorialsDocumentation,Release1
Fromthegure,youcanseethatonlywhen

1
and

2
areaboveaminimumvalue,

min
,itisconideredasa
corner(greenregion).
Code
OpenCVhasafunction,
cv2.goodFeaturesToTrack()
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

simple.jpg

)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
corners
=
cv2
.
goodFeaturesToTrack(gray,
25
,
0.01
,
10
)
corners
=
np
.
int0(corners)
for
i
in
corners:
x,y
=
i
.
ravel()
cv2
.
circle(img,(x,y),
3
,
255
,
-
1
)
164Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
plt
.
imshow(img),plt
.
show()
Seetheresultbelow:
Thisfunctionismoreappropriatefortracking.Wewillseethatwhenitstimecomes.
AdditionalResources
Exercises
IntroductiontoSIFT(Scale-InvariantFeatureTransform)
Goal
Inthischapter,

WewilllearnabouttheconceptsofSIFTalgorithm

WewilllearntondSIFTKeypointsandDescriptors.
Theory
OpenCV-PythonTutorialsDocumentation,Release1
So,in2004,
D.Lowe
,UniversityofBritishColumbia,cameupwithanewalgorithm,ScaleInvariantFeatureTrans-
form(SIFT)inhispaper,
DistinctiveImageFeaturesfromScale-InvariantKeypoints
,whichextractkeypointsand
computeitsdescriptors.
(ThispaperiseasytounderstandandconsideredtobebestmaterialavailableonSIFT.So
thisexplanationisjustashortsummaryofthispaper)
.
TherearemainlyfourstepsinvolvedinSIFTalgorithm.Wewillseethemone-by-one.

166Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
OncethisDoGarefound,imagesaresearchedforlocalextremaoverscaleandspace.Foreg,onepixelinanimageis
comparedwithits8neighboursaswellas9pixelsinnextscaleand9pixelsinpreviousscales.Ifitisalocalextrema,
itisapotentialkeypoint.Itbasicallymeansthatkeypointisbestrepresentedinthatscale.Itisshowninbelowimage:
2
OpenCV-PythonTutorialsDocumentation,Release1
2.KeypointLocalization
import
cv2
import
numpy
as
np
img
=
cv2
.
imread(

home.jpg

)
168Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
sift
=
cv2
.
SIFT()
kp
=
sift
.
img
=
cv2
.
drawKeypoints(gray,kp,flags
=
cv2
.
DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
cv2
.
imwrite(

sift_keypoints.jpg

,img)
Seethetworesultsbelow:
OpenCV-PythonTutorialsDocumentation,Release1
sift
=
cv2
.
SIFT()
kp,des
=
sift
.
Herekpwillbealistofkeypointsanddesisanumpyarrayofshape
Number
_
of
_
Keypoints

128
.
170Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
OpenCV-PythonTutorialsDocumentation,Release1
Inshort,SURFaddsalotoffeaturestoimprovethespeedineverystep.Analysisshowsitis3timesfasterthanSIFT
whileperformanceiscomparabletoSIFT.SURFisgoodathandlingimageswithblurringandrotation,butnotgood
athandlingviewpointchangeandilluminationchange.
SURFinOpenCV
OpenCVprovidesSURFfunctionalitiesjustlikeSIFT.YouinitiateaSURFobjectwithsomeoptionalconditionslike
���
img
=
cv2
.
imread(

fly.png

,
0
)
#CreateSURFobject.Youcanspecifyparamshereorlater.
172Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
���
len
(kp)
699
1199keypointsistoomuchtoshowinapicture.Wereduceittosome50todrawitonanimage.Whilematching,we
mayneedallthosefeatures,butnotnow.SoweincreasetheHessianThreshold.
#CheckpresentHessianthreshold
��

print
surf
.
hessianThreshold
400.0
���
img2
=
cv2
.
drawKeypoints(img,kp,
None
,(
255
,
0
,
0
),
4
)
���
plt
.
imshow(img2),plt
.
show()
NowIwanttoapplyU-SURF,sothatitwon'tndtheorientation.
OpenCV-PythonTutorialsDocumentation,Release1
Seetheresultsbelow.Alltheorientationsareshowninsamedirection.Itismorefasterthanprevious.Ifyouare
Finallywecheckthedescriptorsizeandchangeitto128ifitisonly64-dim.
#Findsizeofdescriptor
��

print
surf
.
descriptorSize()
64
#Thatmeansflag,"extended"isFalse.
��

surf
.
extended
False
174Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
��

print
des
.
shape
(
47
,
128
)
Remainingpartismatchingwhichwewilldoinanotherchapter.
AdditionalResources
Exercises
OpenCV-PythonTutorialsDocumentation,Release1
4.
Nowthepixel
p
176Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
5.
Dependingonthesestates,thefeaturevector
P
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

simple.jpg

,
0
)
#InitiateFASTobjectwithdefaultvalues
OpenCV-PythonTutorialsDocumentation,Release1
fast
=
cv2
.
Seetheresults.FirstimageshowsFASTwithnonmaxSuppressionandsecondonewithoutnonmaxSuppression:
AdditionalResources
1.
178Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Goal
Inthischapter

WewillseethebasicsofBRIEFalgorithm
Theory
WeknowSIFTuses128-dimvectorfordescriptors.Sinceitisusingoatingpointnumbers,ittakesbasically512
bytes.SimilarlySURFalsotakesminimumof256bytes(for64-dim).Creatingsuchavectorforthousandsof
featurestakesalotofmemorywhicharenotfeasibleforresouce-constraintapplicationsespeciallyforembedded
systems.Largerthememory,longerthetimeittakesformatching.
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

simple.jpg

,
0
)
OpenCV-PythonTutorialsDocumentation,Release1
#computethedescriptorswithBRIEF
kp,des
=
brief
.
compute(img,kp)
print
brief
.
Thefunction
180Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
binaryteststondtheonesthathavebothhighvarianceandmeanscloseto0.5,aswellasbeinguncorrelated.The
resultiscalled
rBRIEF
.
Fordescriptormatching,multi-probeLSHwhichimprovesonthetraditionalLSH,isused.ThepapersaysORBis
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

simple.jpg

,
0
)
Seetheresultbelow:
OpenCV-PythonTutorialsDocumentation,Release1
ORBfeaturematching,wewilldoinanotherchapter.
AdditionalResources
1.
182Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
ForBFmatcher,rstwehavetocreatetheBFMatcherobjectusing
cv2.BFMatcher()
.Ittakestwooptionalparams.
Firstoneis
normType
.Itspeciesthedistancemeasurementtobeused.Bydefault,itis
cv2.NORM_L2
.Itisgood
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img1
=
cv2
.
imread(

box.png

,
0
)
#queryImage
img2
=
cv2
.
imread(

box_in_scene.png

,
0
)
#trainImage
NextwecreateaBFMatcherobjectwithdistancemeasurement
cv2.NORM_HAMMING
(sinceweareusingORB)and
crossCheck
#createBFMatcherobject
bf
=
cv2
.
BFMatcher(cv2
.
NORM_HAMMING,crossCheck
=
True
)
#Matchdescriptors.
matches
=
bf
.
match(des1,des2)
#Sortthemintheorderoftheirdistance.
matches
=
sorted
(matches,key
=
lambda
x:x
.
distance)
OpenCV-PythonTutorialsDocumentation,Release1
#Drawfirst10matches.
img3
=
cv2
.
drawMatches(img1,kp1,img2,kp2,matches[:
10
],flags
=
2
)
plt
.
imshow(img3),plt
.
show()
BelowistheresultIgot:
WhatisthisMatcherObject?
Theresultof
matches=bf.match(des1,des2)
lineisalistofDMatchobjects.ThisDMatchobjecthas
followingattributes:

DMatch.distance
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
184Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
img1
=
cv2
.
imread(

box.png

,
0
)
#queryImage
img2
=
cv2
.
imread(

box_in_scene.png

,
0
)
#trainImage
Seetheresultbelow:
FLANNbasedMatcher
FLANNstandsforFastLibraryforApproximateNearestNeighbors.Itcontainsacollectionofalgorithmsoptimized
OpenCV-PythonTutorialsDocumentation,Release1
index_params
=
dict
(algorithm
=
FLANN_INDEX_KDTREE,trees
=
5
)
WhileusingORB,youcanpassthefollowing.Thecommentedvaluesarerecommendedasperthedocs,butitdidn't
providerequiredresultsinsomecases.Othervaluesworkedne.:
index_params
=
dict
(algorithm
=
FLANN_INDEX_LSH,
table_number
=
6
,
#12
key_size
=
12
,
#20
multi_probe_level
=
1
)
#2
SeconddictionaryistheSearchParams.Itspeciesthenumberoftimesthetreesintheindexshouldberecursively
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img1
=
cv2
.
imread(

box.png

,
0
)
#queryImage
img2
=
cv2
.
imread(

box_in_scene.png

,
0
)
#trainImage
Seetheresultbelow:
186Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
FeatureMatching+HomographytondObjects
Goal
Inthischapter,

WewillmixupthefeaturematchingandndHomographyfromcalib3dmoduletondknownobjectsin
acompleximage.
Basics
Sowhatwedidinlastsession?WeusedaqueryImage,foundsomefeaturepointsinit,wetookanothertrainImage,
foundthefeaturesinthatimagetooandwefoundthebestmatchesamongthem.Inshort,wefoundlocationsofsome
partsofanobjectinanotherclutteredimage.ThisinformationissufcienttondtheobjectexactlyonthetrainImage.
Forthat,wecanuseafunctionfromcalib3dmodule,ie
cv2.ndHomography()
OpenCV-PythonTutorialsDocumentation,Release1
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
MIN_MATCH_COUNT
=
10
img1
=
cv2
.
imread(

box.png

,
0
)
#queryImage
img2
=
cv2
.
imread(

box_in_scene.png

,
0
)
#trainImage
if
len
(good)

MIN_MATCH_COUNT:
src_pts
=
np
.
float32([kp1[m
.
queryIdx]
.
pt
for
m
in
good])
.
reshape(
-
1
,
1
,
2
)
dst_pts
=
np
.
float32([kp2[m
.
trainIdx]
.
pt
for
m
in
good])
.
reshape(
-
1
,
1
,
2
)
M,mask
=
cv2
.
findHomography(src_pts,dst_pts,cv2
.
RANSAC,
5.0
)
matchesMask
=
mask
.
ravel()
.
tolist()
h,w
=
img1
.
shape
pts
=
np
.
float32([[
0
,
0
],[
0
,h
-
1
],[w
-
1
,h
-
1
],[w
-
1
,
0
]])
.
reshape(
-
1
,
1
,
2
)
dst
=
cv2
.
Finallywedrawourinliers(ifsuccessfullyfoundtheobject)ormatchingkeypoints(iffailed).
draw_params
=
dict
(matchColor
=
(
0
,
255
,
0
),
#drawmatchesingreencolor
singlePointColor
=
None
,
188Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
matchesMask
=
matchesMask,
#drawonlyinliers
flags
=
2
)
img3
=
cv2
.
drawMatches(img1,kp1,img2,kp2,good,
None
,
*
*
draw_params)
plt
.
imshow(img3,

gray

),plt
.
show()
Seetheresultbelow.Objectismarkedinwhitecolorinclutteredimage:
AdditionalResources
Exercises
VideoAnalysis

MeanshiftandCamshift
Wehavealreadyseenanexampleofcolor-basedtracking.Itissimpler.

OpticalFlow
1.6.VideoAnalysis
189
OpenCV-PythonTutorialsDocumentation,Release1

BackgroundSubtraction
Inseveralapplications,weneedtoextractforegroundforfurtheroperations
190Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
MeanshiftandCamshift
Goal
Inthischapter,

WewilllearnaboutMeanshiftandCamshiftalgorithmstondandtrackobjectsinvideos.
Meanshift
Theinitialwindowisshowninbluecirclewiththename“C1”.Itsoriginalcenterismarkedinbluerectangle,named
withgreencircle,named“C2”.Asyoucanseeinimage,ithasmaximumnumberofpoints.Thewholeprocessis
demonstratedonastaticimagebelow:
1.6.VideoAnalysis
191
OpenCV-PythonTutorialsDocumentation,Release1
MeanshiftinOpenCV
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(

slow.flv

)
#takefirstframeofthevideo
ThreeframesinavideoIusedisgivenbelow:
192Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
1.6.VideoAnalysis
193
OpenCV-PythonTutorialsDocumentation,Release1
Camshift
Didyoucloselywatchthelastresult?Thereisaproblem.Ourwindowalwayshasthesamesizewhencarisfarther
awayanditisveryclosetocamera.Thatisnotgood.Weneedtoadaptthewindowsizewithsizeandrotationof
M
00
256
.Italso
calculatestheorientationofbestttingellipsetoit.Againitappliesthemeanshiftwithnewscaledsearchwindow
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(

slow.flv

)
#takefirstframeofthevideo
194Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
k
=
cv2
.
waitKey(
60
)
&
0xff
if
k
==
27
:
break
else
:
cv2
.
imwrite(
chr
(k)
+
"
.jpg
"
,img2)
else
:
break
cv2
.
destroyAllWindows()
cap
.
release()
Threeframesoftheresultisshownbelow:
1.6.VideoAnalysis
195
OpenCV-PythonTutorialsDocumentation,Release1
196Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
FrenchWikipediapageon
Camshift
.(Thetwoanimationsaretakenfromhere)
2.
Bradski,G.R.,“Realtimefaceandobjecttrackingasacomponentofaperceptualuserinterface,”Applications
ofComputerVision,1998.WACV`98.Proceedings.,FourthIEEEWorkshopon,vol.,no.,pp.214,219,19-21
Oct1998
Exercises
1.
OpenCVcomeswithaPythonsampleoninteractivedemoofcamshift.Useit,hackit,understandit.
OpticalFlow
Goal
Inthischapter,

Itshowsaballmovingin5consecutiveframes.Thearrowshowsitsdisplacementvector.Opticalowhasmany
applicationsinareaslike:

StructurefromMotion

VideoCompression

VideoStabilization...
Opticalowworksonseveralassumptions:
1.
1.6.VideoAnalysis
197
OpenCV-PythonTutorialsDocumentation,Release1
2.
Neighbouringpixelshavesimilarmotion.
Considerapixel
I
(
x;y;t
)
inrstframe(Checkanewdimension,time,isaddedhere.Earlierwewereworkingwith
imagesonly,sononeedoftime).Itmovesbydistance
(
dx;dy
)
innextframetakenafter
dt
time.Sosincethosepixels
arethesameandintensitydoesnotchange,wecansay,
I
(
x;y;t
)=
I
(
x
+
dx;y
+
dy;t
+
dt
)
Thentaketaylorseriesapproximationofright-handside,removecommontermsanddivideby
dt
@x
;
f
y
=
@f
@x
u
=
dx
dt
;
v
=
dy
dt
AboveequationiscalledOpticalFlowequation.Init,wecannd
f
x
and
f
y
,theyareimagegradients.Similarly
f
t
isthegradientalongtime.But
(
u;v
)
isunknown.Wecannotsolvethisoneequationwithtwounknownvariables.So
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(

slow.flv

)
198Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
(Thiscodedoesn'tcheckhowcorrectarethenextkeypoints.Soevenifanyfeaturepointdisappearsinimage,there
isachancethatopticalowndsthenextpointwhichmaylookclosetoit.Soactuallyforarobusttracking,corner
1.6.VideoAnalysis
199
OpenCV-PythonTutorialsDocumentation,Release1
DenseOpticalFlowinOpenCV
import
cv2
import
numpy
as
np
cap
=
cv2
.
VideoCapture(
"
vtest.avi
"
)
200Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
hsv[
.
.
.
,
0
]
=
ang
*
180
/
np
.
pi
/
2
hsv[
.
.
.
,
2
]
=
cv2
.
normalize(mag,
None
,
0
,
255
,cv2
.
NORM_MINMAX)
rgb
=
cv2
.
cvtColor(hsv,cv2
.
COLOR_HSV2BGR)
cv2
.
imshow(

frame2

,rgb)
k
=
cv2
.
waitKey(
30
)
&
0xff
if
k
==
27
:
break
elif
k
==
ord
(

s

):
cv2
.
imwrite(

opticalfb.png

,frame2)
cv2
.
imwrite(

opticalhsv.png

,rgb)
prvs
=
next
cap
.
release()
cv2
.
destroyAllWindows()
Seetheresultbelow:
1.6.VideoAnalysis
201
OpenCV-PythonTutorialsDocumentation,Release1
OpenCVcomeswithamoreadvancedsampleondenseopticalow,pleasesee
samples/python2/opt_flow.
py
.
202Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
1.
Checkthecodein
samples/python2/lk_track.py
.Trytounderstandthecode.
2.
Checkthecodein
samples/python2/opt_flow.py
.Trytounderstandthecode.
BackgroundSubtraction
Goal
Inthischapter,

caseslikevisitorcounterwhereastaticcameratakesthenumberofvisitorsenteringorleavingtheroom,oratrafc
Severalalgorithmswereintroducedforthispurpose.OpenCVhasimplementedthreesuchalgorithmswhichisvery
easytouse.Wewillseethemone-by-one.
BackgroundSubtractorMOG
ItisaGaussianMixture-basedBackground/ForegroundSegmentationAlgorithm.Itwasintroducedinthepaper“An
.
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(

vtest.avi

)
fgbg
=
cv2
.
createBackgroundSubtractorMOG()
while
(
1
):
1.6.VideoAnalysis
203
OpenCV-PythonTutorialsDocumentation,Release1
(Alltheresultsareshownattheendforcomparison).
BackgroundSubtractorMOG2
ItisalsoaGaussianMixture-basedBackground/ForegroundSegmentationAlgorithm.Itisbasedontwopapersby
Z.Zivkovic,“ImprovedadaptiveGausianmixturemodelforbackgroundsubtraction”in2004and“EfcientAdaptive
DensityEstimationperImagePixelfortheTaskofBackgroundSubtraction”in2006.Oneimportantfeatureofthis
algorithmisthatitselectstheappropriatenumberofgaussiandistributionforeachpixel.(Remember,inlastcase,
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(

vtest.avi

)
fgbg
=
cv2
.
createBackgroundSubtractorMOG2()
while
(
1
):
(Resultsgivenattheend)
BackgroundSubtractorGMG
Thisalgorithmcombinesstatisticalbackgroundimageestimationandper-pixelBayesiansegmentation.Itwasin-
troducedbyAndrewB.Godbehere,AkihiroMatsukawa,KenGoldbergintheirpaper“VisualTrackingofHuman
VisitorsunderVariable-LightingConditionsforaResponsiveAudioArtInstallation”in2012.Asperthepaper,the
204Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
import
numpy
as
np
import
cv2
cap
=
cv2
.
VideoCapture(

vtest.avi

)
kernel
=
cv2
.
while
(
1
):
Results
OriginalFrame
Belowimageshowsthe200thframeofavideo
ResultofBackgroundSubtractorMOG
1.6.VideoAnalysis
205
OpenCV-PythonTutorialsDocumentation,Release1
ResultofBackgroundSubtractorMOG2
Graycolorregionshowsshadowregion.
ResultofBackgroundSubtractorGMG
Noiseisremovedwithmorphologicalopening.
206Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
Exercises
CameraCalibrationand3DReconstruction

CameraCalibration

PoseEstimation
Thisisasmallsectionwhichwillhelpyoutocreatesomecool3Deffects
withcalibmodule.


DepthMapfromStereoImages
Extractdepthinformationfrom2Dimages.
1.7.CameraCalibrationand3DReconstruction207
OpenCV-PythonTutorialsDocumentation,Release1
CameraCalibration
Goal
Inthissection,

208Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Thisdistortionissolvedasfollows:
x
corrected
=
x
(1+
k
1
r
2
+
k
2
r
4
+
k
3
r
6
)
y
corrected
=
y
(1+
k
1
r
2
+
k
2
r
4
+
k
3
r
6
)
Similarly,anotherdistortionisthetangentialdistortionwhichoccursbecauseimagetakinglenseisnotalignedper-
fectlyparalleltotheimagingplane.Sosomeareasinimagemaylooknearerthanexpected.Itissolvedasbelow:
x
corrected
=
x
+[2
p
1
xy
+
p
2
(
r
2
+2
x
2
)]
y
corrected
=
y
+[
p
1
(
r
2
+2
y
2
)+2
p
2
xy
]
1.7.CameraCalibrationand3DReconstruction209
OpenCV-PythonTutorialsDocumentation,Release1
alsocalledcameramatrix.Itdependsonthecameraonly,sooncecalculated,itcanbestoredforfuturepurposes.Itis
expressedasa3x3matrix:
cameramatrix
=
2
4
f
x
0
c
x
0
f
y
c
y
001
3
5
Thesecornerswillbeplacedinanorder(fromleft-to-right,top-to-bottom)
Seealso:
Thisfunctionmaynotbeabletondtherequiredpatterninalltheimages.Soonegoodoptionistowritethecode
suchthat,itstartsthecameraandcheckeachframeforrequiredpattern.Oncepatternisobtained,ndthecorners
andstoreitinalist.Alsoprovidessomeintervalbeforereadingnextframesothatwecanadjustourchessboardin
differentdirection.Continuethisprocessuntilrequirednumberofgoodpatternsareobtained.Evenintheexample
providedhere,wearenotsureoutof14imagesgiven,howmanyaregood.Sowereadalltheimagesandtakethe
goodones.
Seealso:
Insteadofchessboard,wecanusesomecirculargrid,butthenusethefunction
cv2.ndCirclesGrid()
tondthe
pattern.Itissaidthatlessnumberofimagesareenoughwhenusingcirculargrid.
Oncewendthecorners,wecanincreasetheiraccuracyusing
cv2.cornerSubPix()
.Wecanalsodrawthepattern
using
cv2.drawChessboardCorners()
.Allthesestepsareincludedinbelowcode:
210Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
import
numpy
as
np
import
cv2
import
glob
#terminationcriteria
criteria
=
(cv2
.
TERM_CRITERIA_EPS
+
cv2
.
TERM_CRITERIA_MAX_ITER,
30
,
0.001
)
#prepareobjectpoints,like(0,0,0),(1,0,0),(2,0,0)....,(6,5,0)
objp
=
np
.
zeros((
6
*
7
,
3
),np
.
float32)
objp[:,:
2
]
=
np
.
mgrid[
0
:
7
,
0
:
6
]
.
T
.
reshape(
-
1
,
2
)
#Arraystostoreobjectpointsandimagepointsfromalltheimages.
objpoints
=
[]
#3dpointinrealworldspace
imgpoints
=
[]
#2dpointsinimageplane.
images
=
glob
.
glob(

*
.jpg

)
for
fname
in
images:
img
=
cv2
.
imread(fname)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
#Findthechessboardcorners
Oneimagewithpatterndrawnonitisshownbelow:
1.7.CameraCalibrationand3DReconstruction211
OpenCV-PythonTutorialsDocumentation,Release1
Calibration
Sonowwehaveourobjectpointsandimagepointswearereadytogoforcalibration.Forthatweusethefunction,
cv2.calibrateCamera()
Undistortion
212Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
img
=
cv2
.
imread(

left12.jpg

)
h,w
=
img
.
shape[:
2
]
newcameramtx,roi
=
cv2
.
1.Usingcv2.undistort()
Thisistheshortestpath.JustcallthefunctionanduseROIobtainedabovetocroptheresult.
#undistort
dst
=
cv2
.
undistort(img,mtx,dist,
None
,newcameramtx)
#croptheimage
x,y,w,h
=
roi
dst
=
dst[y:y
+
h,x:x
+
w]
cv2
.
imwrite(

calibresult.png

,dst)
2.Usingremapping
Thisiscurvedpath.Firstndamappingfunctionfromdistortedimagetoundistortedimage.Thenusetheremap
function.
#undistort
mapx,mapy
=
cv2
.
initUndistortRectifyMap(mtx,dist,
None
,newcameramtx,(w,h),
5
)
dst
=
cv2
.
remap(img,mapx,mapy,cv2
.
INTER_LINEAR)
#croptheimage
x,y,w,h
=
roi
dst
=
dst[y:y
+
h,x:x
+
w]
cv2
.
imwrite(

calibresult.png

,dst)
1.7.CameraCalibrationand3DReconstruction213
OpenCV-PythonTutorialsDocumentation,Release1
Youcanseeintheresultthatalltheedgesarestraight.
mean_error
=
0
for
i
in
xrange(
len
(objpoints)):
imgpoints2,_
=
cv2
.
projectPoints(objpoints[i],rvecs[i],tvecs[i],mtx,dist)
error
=
cv2
.
norm(imgpoints[i],imgpoints2,cv2
.
NORM_L2)
/
len
(imgpoints2)
tot_error
+
=
error
print
"
totalerror:
"
,mean_error
/
len
(objpoints)
AdditionalResources
Exercises
1.
Trycameracalibrationwithcirculargrid.
214Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
PoseEstimation
Goal
Inthissection,

Wewilllearntoexploitcalib3dmoduletocreatesome3Deffectsinimages.
Basics
Thisisgoingtobeasmallsection.Duringthelastsessiononcameracalibration,youhavefoundthecameramatrix,
import
cv2
import
numpy
as
np
import
glob
#Loadpreviouslysaveddata
with
np
.
load(

B.npz

)
as
X:
mtx,dist,_,_
=
[X[i]
for
i
in
(

mtx

,

dist

,

rvecs

,

tvecs

)]
cv2.ndChessboardCorners()
)and
axispoints
todrawa3Daxis.
def
draw
(img,corners,imgpts):
corner
=
tuple
(corners[
0
]
.
ravel())
img
=
cv2
.
line(img,corner,
tuple
(imgpts[
0
]
.
ravel()),(
255
,
0
,
0
),
5
)
img
=
cv2
.
line(img,corner,
tuple
(imgpts[
1
]
.
ravel()),(
0
,
255
,
0
),
5
)
img
=
cv2
.
line(img,corner,
tuple
(imgpts[
2
]
.
ravel()),(
0
,
0
,
255
),
5
)
Thenasinpreviouscase,wecreateterminationcriteria,objectpoints(3Dpointsofcornersinchessboard)andaxis
points.Axispointsarepointsin3Dspacefordrawingtheaxis.Wedrawaxisoflength3(unitswillbeintermsof
chesssquaresizesincewecalibratedbasedonthatsize).SoourXaxisisdrawnfrom(0,0,0)to(3,0,0),soforYaxis.
ForZaxis,itisdrawnfrom(0,0,0)to(0,0,-3).Negativedenotesitisdrawntowardsthecamera.
criteria
=
(cv2
.
TERM_CRITERIA_EPS
+
cv2
.
TERM_CRITERIA_MAX_ITER,
30
,
0.001
)
objp
=
np
.
zeros((
6
*
7
,
3
),np
.
float32)
objp[:,:
2
]
=
np
.
mgrid[
0
:
7
,
0
:
6
]
.
T
.
reshape(
-
1
,
2
)
axis
=
np
.
float32([[
3
,
0
,
0
],[
0
,
3
,
0
],[
0
,
0
,
-
3
]])
.
reshape(
-
1
,
3
)
Now,asusual,weloadeachimage.Searchfor7x6grid.Iffound,wereneitwithsubcornerpixels.Thentocalculate
therotationandtranslation,weusethefunction,
cv2.solvePnPRansac()
.Oncewethosetransformationmatrices,
weusethemtoprojectour
axispoints
totheimageplane.Insimplewords,wendthepointsonimageplane
1.7.CameraCalibrationand3DReconstruction215
OpenCV-PythonTutorialsDocumentation,Release1
for
fname
in
glob
.
glob(

left
*
.jpg

):
img
=
cv2
.
imread(fname)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
Seesomeresultsbelow.Noticethateachaxisis3squareslong.:
216Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
RenderaCube
Ifyouwanttodrawacube,modifythedraw()functionandaxispointsasfollows.
Modieddraw()function:
def
draw
(img,corners,imgpts):
imgpts
=
np
.
int32(imgpts)
.
reshape(
-
1
,
2
)
#drawgroundflooringreen
img
=
cv2
.
drawContours(img,[imgpts[:
4
]],
-
1
,(
0
,
255
,
0
),
-
3
)
#drawpillarsinbluecolor
for
i,j
in
zip
(
range
(
4
),
range
(
4
,
8
)):
img
=
cv2
.
line(img,
tuple
(imgpts[i]),
tuple
(imgpts[j]),(
255
),
3
)
#drawtoplayerinredcolor
img
=
cv2
.
drawContours(img,[imgpts[
4
:]],
-
1
,(
0
,
0
,
255
),
3
)
1.7.CameraCalibrationand3DReconstruction217
OpenCV-PythonTutorialsDocumentation,Release1
Modiedaxispoints.Theyarethe8cornersofacubein3Dspace:
axis
=
np
.
float32([[
0
,
0
,
0
],[
0
,
3
,
0
],[
3
,
3
,
0
],[
3
,
0
,
0
],
[
0
,
0
,
-
3
],[
0
,
3
,
-
3
],[
3
,
3
,
-
3
],[
3
,
0
,
-
3
]])
Andlookattheresultbelow:
218Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Ifweareusingonlytheleftcamera,wecan'tndthe3Dpointcorrespondingtothepoint
x
inimagebecauseevery
pointontheline
OX
projectstothesamepointontheimageplane.Butconsidertherightimagealso.Nowdifferent
pointsontheline
OX
projectstodifferentpoints(
x
0
)inrightplane.Sowiththesetwoimages,wecantriangulatethe
correct3Dpoint.Thisisthewholeidea.
Theprojectionofthedifferentpointson
OX
formalineonrightplane(line
l
0
).Wecallit
epiline
correspondingto
thepoint
x
.Itmeans,tondthepoint
x
ontherightimage,searchalongthisepiline.Itshouldbesomewhereonthis
line(Thinkofitthisway,tondthematchingpointinotherimage,youneednotsearchthewholeimage,justsearch
1.7.CameraCalibrationand3DReconstruction219
OpenCV-PythonTutorialsDocumentation,Release1
Butweprefermeasurementstobedoneinpixelcoordinates,right?FundamentalMatrixcontainsthesameinformation
asEssentialMatrixinadditiontotheinformationabouttheintrinsicsofbothcamerassothatwecanrelatethetwo
camerasinpixelcoordinates.(Ifweareusingrectiedimagesandnormalizethepointbydividingbythefocallengths,
F
=
E
).Insimplewords,FundamentalMatrixF,mapsapointinoneimagetoaline(epiline)intheotherimage.
Thisiscalculatedfrommatchingpointsfromboththeimages.Aminimumof8suchpointsarerequiredtondthe
import
cv2
import
numpy
as
np
from
matplotlib
import
pyplot
as
plt
img1
=
cv2
.
imread(

myleft.jpg

,
0
)
#queryimage#leftimage
img2
=
cv2
.
imread(

myright.jpg

,
0
)
#trainimage#rightimage
sift
=
cv2
.
SIFT()
#findthekeypointsanddescriptorswithSIFT
kp1,des1
=
sift
.
220Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
pts2
=
[]
#ratiotestasperLowespaper
for
i,(m,n)
in
enumerate
(matches):
if
m
.
distance

0.8
*
n
.
distance:
good
.
append(m)
pts2
.
append(kp2[m
.
trainIdx]
.
pt)
pts1
.
append(kp1[m
.
queryIdx]
.
pt)
pts1
=
np
.
int32(pts1)
pts2
=
np
.
int32(pts2)
F,mask
=
cv2
.
findFundamentalMat(pts1,pts2,cv2
.
FM_LMEDS)
#Weselectonlyinlierpoints
pts1
=
pts1[mask
.
ravel()
==
1
]
pts2
=
pts2[mask
.
ravel()
==
1
]
Nextwendtheepilines.Epilinescorrespondingtothepointsinrstimageisdrawnonsecondimage.Somentioning
def
drawlines
(img1,img2,lines,pts1,pts2):
img1-imageonwhichwedrawtheepilinesforthepointsinimg2
lines-correspondingepilines
r,c
=
img1
.
shape
img1
=
cv2
.
cvtColor(img1,cv2
.
COLOR_GRAY2BGR)
img2
=
cv2
.
cvtColor(img2,cv2
.
COLOR_GRAY2BGR)
for
r,pt1,pt2
in
zip
(lines,pts1,pts2):
color
=
tuple
(np
.
random
.
randint(
0
,
255
,
3
)
.
tolist())
x0,y0
=
map
(
int
,[
0
,
-
r[
2
]
/
r[
1
]])
x1,y1
=
map
(
int
,[c,
-
(r[
2
]
+
r[
0
]
*
c)
/
r[
1
]])
img1
=
cv2
.
line(img1,(x0,y0),(x1,y1),color,
1
)
img1
=
cv2
.
circle(img1,
tuple
(pt1),
5
,color,
-
1
)
img2
=
cv2
.
circle(img2,
tuple
(pt2),
5
,color,
-
1
)
Nowwendtheepilinesinboththeimagesanddrawthem.
#Findepilinescorrespondingtopointsinrightimage(secondimage)and
#drawingitslinesonleftimage
lines1
=
cv2
.
computeCorrespondEpilines(pts2
.
reshape(
-
1
,
1
,
2
),
2
,F)
lines1
=
lines1
.
reshape(
-
1
,
3
)
img5,img6
=
drawlines(img1,img2,lines1,pts1,pts2)
#Findepilinescorrespondingtopointsinleftimage(firstimage)and
#drawingitslinesonrightimage
lines2
=
cv2
.
computeCorrespondEpilines(pts1
.
reshape(
-
1
,
1
,
2
),
1
,F)
lines2
=
lines2
.
reshape(
-
1
,
3
)
img3,img4
=
drawlines(img2,img1,lines2,pts2,pts1)
plt
.
subplot(
121
),plt
.
imshow(img5)
plt
.
subplot(
122
),plt
.
imshow(img3)
plt
.
show()
1.7.CameraCalibrationand3DReconstruction221
OpenCV-PythonTutorialsDocumentation,Release1
222Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
pointistheepipole.
1.7.CameraCalibrationand3DReconstruction223
OpenCV-PythonTutorialsDocumentation,Release1
Theabovediagramcontainsequivalenttriangles.Writingtheirequivalentequationswillyieldusfollowingresult:
disparity
=
x

x
0
=
Bf
Z
x
and
x
0
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
imgL
=
cv2
.
imread(

tsukuba_l.png

,
0
)
imgR
=
cv2
.
imread(

tsukuba_r.png

,
0
)
224Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
stereo
=
cv2
.
createStereoBM(numDisparities
=
16
,blockSize
=
15
)
disparity
=
stereo
.
compute(imgL,imgR)
plt
.
imshow(disparity,

gray

)
plt
.
show()
Belowimagecontainstheoriginalimage(left)anditsdisparitymap(right).Asyoucansee,resultiscontaminated
Note:
AdditionalResources
Exercises
1.
OpenCVsamplescontainanexampleofgeneratingdisparitymapandits3Dreconstruction.Check
stereo_match.py
inOpenCV-Pythonsamples.
MachineLearning

K-NearestNeighbour
LearntousekNNforclassicationPluslearnabouthandwrittendigitrecog-
nitionusingkNN

SupportVectorMachines(SVM)
1.8.MachineLearning
225
OpenCV-PythonTutorialsDocumentation,Release1
UnderstandconceptsofSVM

K-MeansClustering
LearntouseK-MeansClusteringtogroupdatatoanumberofclusters.Plus
learntodocolorquantizationusingK-MeansClustering
226Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
K-NearestNeighbour

Understandingk-NearestNeighbour

OCRofHand-writtenDatausingkNN
1.8.MachineLearning
227
OpenCV-PythonTutorialsDocumentation,Release1
Understandingk-NearestNeighbour
Goal
Inthischapter,wewillunderstandtheconceptsofk-NearestNeighbour(kNN)algorithm.
Theory
kNNisoneofthesimplestofclassicationalgorithmsavailableforsupervisedlearning.Theideaistosearchfor
closestmatchofthetestdatainfeaturespace.Wewilllookintoitwithbelowimage.
Intheimage,therearetwofamilies,
BlueSquaresandRedTriangles
.Wecalleachfamilyas
Class
.Theirhousesare
shownintheirtownmapwhichwecall
featurespace
.
(Youcanconsiderafeaturespaceasaspacewherealldatas
areprojected.Forexample,considera2Dcoordinatespace.Eachdatahastwofeatures,xandycoordinates.Youcan
representthisdatainyour2Dcoordinatespace,right?Nowimagineiftherearethreefeatures,youneed3Dspace.
NowconsiderNfeatures,whereyouneedN-dimensionalspace,right?ThisN-dimensionalspaceisitsfeaturespace.
Inourimage,youcanconsideritasa2Dcasewithtwofeatures)
.
Nowanewmembercomesintothetownandcreatesanewhome,whichisshownasgreencircle.Heshouldbeadded
tooneoftheseBlue/Redfamilies.Wecallthatprocess,
Classication
.Whatwedo?SincewearedealingwithkNN,
228Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
other2Bluefamilies.SoheismoreeligibletobeaddedtoRed.Sohowdowemathematicallyexplainthat?Wegive
import
cv2
import
numpy
as
np
import
matplotlib
.
pyplot
as
plt
1.8.MachineLearning
229
OpenCV-PythonTutorialsDocumentation,Release1
2.
Thelabelsofk-NearestNeighbours.
3.
Correspondingdistancesfromnew-comertoeachnearestneighbour.
newcomer
=
np
.
random
.
randint(
0
,
100
,(
1
,
2
))
.
astype(np
.
float32)
plt
.
scatter(newcomer[:,
0
],newcomer[:,
1
],
80
,

g

,

o

)
knn
=
cv2
.
KNearest()
knn
.
train(trainData,responses)
Igottheresultasfollows:
result:[[
1.
]]
neighbours:[[
1.
1.
1.
]]
distance:[[
53.
58.
61.
]]
Itsaysournew-comergot3neighbours,allfromBluefamily.Therefore,heislabelledasBluefamily.Itisobvious
fromplotbelow:
Ifyouhavelargenumberofdata,youcanjustpassitasarray.Correspondingresultsarealsoobtainedasarrays.
230Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
#10newcomers
newcomers
=
np
.
random
.
randint(
0
,
100
,(
10
,
2
))
.
astype(np
.
float32)
AdditionalResources
1.
NPTELnotesonPatternRecognition,Chapter11
Exercises
OCRofHand-writtenDatausingkNN
Goal
Inthischapter

WewilluseourknowledgeonkNNtobuildabasicOCRapplication.

import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

digits.png

)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
#Nowwesplittheimageto5000cells,each20x20size
cells
=
[np
.
hsplit(row,
100
)
for
row
in
np
.
vsplit(gray,
50
)]
#MakeitintoaNumpyarray.Itsizewillbe(50,100,20,20)
x
=
np
.
array(cells)
#Nowwepreparetrain_dataandtest_data.
train
=
x[:,:
50
]
.
reshape(
-
1
,
400
)
.
astype(np
.
float32)
#Size=(2500,400)
test
=
x[:,
50
:
100
]
.
reshape(
-
1
,
400
)
.
astype(np
.
float32)
#Size=(2500,400)
#Createlabelsfortrainandtestdata
k
=
np
.
arange(
10
)
train_labels
=
np
.
repeat(k,
250
)[:,np
.
newaxis]
test_labels
=
train_labels
.
copy()
#InitiatekNN,trainthedata,thentestitwithtestdatafork=1
1.8.MachineLearning
231
OpenCV-PythonTutorialsDocumentation,Release1
knn
=
cv2
.
KNearest()
knn
.
train(train,train_labels)
SoourbasicOCRappisready.Thisparticularexamplegavemeanaccuracyof91%.Oneoptionimproveaccuracy
istoaddmoredatafortraining,especiallythewrongones.SoinsteadofndingthistrainingdataeverytimeIstart
#savethedata
np
.
savez(

knn_data.npz

,train
=
train,train_labels
=
train_labels)
#Nowloadthedata
with
np
.
load(

knn_data.npz

)
as
data:
print
data
.
files
train
=
data[

train

]
train_labels
=
data[

train_labels

]
Inmysystem,ittakesaround4.4MBofmemory.Sinceweareusingintensityvalues(uint8data)asfeatures,itwould
from
UCIMachineLearningRepository
import
cv2
import
numpy
as
np
import
matplotlib
.
pyplot
as
plt
232Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
knn
=
cv2
.
KNearest()
knn
.
train(trainData,responses)
Itgivesmeanaccuracyof93.22%.Again,ifyouwanttoincreaseaccuracy,youcaniterativelyadderrordataineach
level.
AdditionalResources
Exercises
SupportVectorMachines(SVM)

UnderstandingSVM

OCRofHand-writtenDatausingSVM
1.8.MachineLearning
233
OpenCV-PythonTutorialsDocumentation,Release1
UnderstandingSVM
Goal
Inthischapter

WewillseeanintuitiveunderstandingofSVM
Theory
LinearlySeparableData
Considertheimagebelowwhichhastwotypesofdata,redandblue.InkNN,foratestdata,weusedtomeasureits
distancetoallthetrainingsamplesandtaketheonewithminimumdistance.Ittakesplentyoftimetomeasureallthe
distancesandplentyofmemorytostoreallthetraining-samples.Butconsideringthedatagiveninimage,shouldwe
needthatmuch?
Consideranotheridea.Wendaline,
f
(
x
)=
ax
1
+
bx
2
+
c
whichdividesboththedatatotworegions.Whenwe
SotondthisDecisionBoundary,youneedtrainingdata.Doyouneedall?NO.Justtheoneswhichareclosetothe
oppositegrouparesufcient.Inourimage,theyaretheonebluelledcircleandtworedlledsquares.Wecancall
them
SupportVectors
andthelinespassingthroughthemarecalled
SupportPlanes
.Theyareadequatefornding
ourdecisionboundary.Weneednotworryaboutallthedata.Ithelpsindatareduction.
Whathappenedis,rsttwohyperplanesarefoundwhichbestrepresentsthedata.Foreg,bluedataisrepresentedby
w
T
x
+
b
0

1
whilereddataisrepresentedby
w
T
x
+
b
0


1
where
w
is
weightvector
(
w
=[
w
1
;w
2
;:::;w
n
]
)and
x
isthefeaturevector(
x
=[
x
1
;x
2
;:::;x
n
]
).
b
0
isthe
bias
.Weightvectordecidestheorientationofdecisionboundary
jj
w
jj
.Marginistwicethisdistance,andweneedtomaximizethismargin.i.e.weneedto
234Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
minimizeanewfunction
L
(
w;b
0
)
withsomeconstraintswhichcanexpressedbelow:
min
w;b
0
L
(
w;b
0
)=
1
2
jj
w
jj
2
subjectto
t
i
(
w
T
x
+
b
0
)

1
8
i
where
t
i
isthelabelofeachclass,
t
i
2
[

1
;
1]
.
Non-LinearlySeparableData
Considersomedatawhichcan'tbedividedintotwowithastraightline.Forexample,consideranone-dimensional
2
p
1
p
2
)

(
q
)=(
q
2
1
;q
2
2
;
p
2
q
1
q
2
)
2
p
1
p
2
)
:
(
q
2
1
;q
2
2
;
p
2
q
1
q
2
)
=
p
2
1
q
2
1
+
p
2
2
q
2
2
+2
p
1
q
1
p
2
q
2
=(
p
1
q
1
+
p
2
q
2
)
2

(
p
)
:
(
q
)=(
p:q
)
2
Itmeans,adotproductinthree-dimensionalspacecanbeachievedusingsquareddotproductintwo-dimensional
space.Thiscanbeappliedtohigherdimensionalspace.Sowecancalculatehigherdimensionalfeaturesfromlower
1.8.MachineLearning
235
OpenCV-PythonTutorialsDocumentation,Release1
Sothenewoptimizationproblemis:
min
w;b
0
L
(
w;b
0
)=
jj
w
jj
2
+
C
X
i

i
subjectto
y
i
(
w
T
x
i
+
b
0
)

1


i
and

i

0
8
i
236Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Here,beforendingtheHOG,wedeskewtheimageusingitssecondordermoments.Sowerstdeneafunction
deskew()
whichtakesadigitimageanddeskewit.Belowisthedeskew()function:
def
deskew
(img):
m
=
cv2
.
moments(img)
if
abs
(m[

mu02

])

1e-2
:
Belowimageshowsabovedeskewfunctionappliedtoanimageofzero.Leftimageistheoriginalimageandright
imageisthedeskewedimage.
NextwehavetondtheHOGDescriptorofeachcell.Forthat,wendSobelderivativesofeachcellinXandY
direction.Thenndtheirmagnitudeanddirectionofgradientateachpixel.Thisgradientisquantizedto16integer
values.Dividethisimagetofoursub-squares.Foreachsub-square,calculatethehistogramofdirection(16bins)
weightedwiththeirmagnitude.Soeachsub-squaregivesyouavectorcontaining16values.Foursuchvectors(of
def
hog
(img):
gx
=
cv2
.
Sobel(img,cv2
.
CV_32F,
1
,
0
)
gy
=
cv2
.
Sobel(img,cv2
.
CV_32F,
0
,
1
)
mag,ang
=
cv2
.
cartToPolar(gx,gy)
#quantizingbinvaluesin(0...16)
bins
=
np
.
int32(bin_n
*
ang
/
(
2
*
np
.
pi))
#Divideto4sub-squares
bin_cells
=
bins[:
10
,:
10
],bins[
10
:,:
10
],bins[:
10
,
10
:],bins[
10
:,
10
:]
mag_cells
=
mag[:
10
,:
10
],mag[
10
:,:
10
],mag[:
10
,
10
:],mag[
10
:,
10
:]
hists
=
[np
.
bincount(b
.
ravel(),m
.
ravel(),bin_n)
for
b,m
in
zip
(bin_cells,mag_
,
!
cells)]
hist
=
np
.
hstack(hists)
1.8.MachineLearning
237
OpenCV-PythonTutorialsDocumentation,Release1
import
cv2
import
numpy
as
np
SZ
=
20
bin_n
=
16
#Numberofbins
svm_params
=
dict
(kernel_type
=
cv2
.
SVM_LINEAR,
svm_type
=
cv2
.
SVM_C_SVC,
C
=
2.67
,gamma
=
5.383
)
affine_flags
=
cv2
.
WARP_INVERSE_MAP
|
cv2
.
INTER_LINEAR
def
deskew
(img):
m
=
cv2
.
moments(img)
if
abs
(m[

mu02

])

1e-2
:
238Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
deskewed
=
[
map
(deskew,row)
for
row
in
test_cells]
hogdata
=
[
map
(hog,row)
for
row
in
deskewed]
testData
=
np
.
float32(hogdata)
.
reshape(
-
1
,bin_n
*
4
)
result
=
svm
.
predict_all(testData)
#######CheckAccuracy########################
mask
=
result
==
responses
correct
=
np
.
count_nonzero(mask)
print
correct
*
100.0
/
result
.
size

K-MeansClusteringinOpenCV
1.8.MachineLearning
239
OpenCV-PythonTutorialsDocumentation,Release1
UnderstandingK-MeansClustering
Goal
Companycan'tcreatet-shirtswithallthesizes.Instead,theydividepeopletoSmall,MediumandLarge,andmanu-
factureonlythese3modelswhichwilltintoallthepeople.Thisgroupingofpeopleintothreegroupscanbedone
byk-meansclustering,andalgorithmprovidesusbest3sizes,whichwillsatisfyallthepeople.Andifitdoesn't,
companycandividepeopletomoregroups,maybeve,andsoon.Checkimagebelow:
240Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Howdoesitwork?
Thisalgorithmisaniterativeprocess.Wewillexplainitstep-by-stepwiththehelpofimages.
Step:1
-Algorithmrandomlychoosestwocentroids,
C
1
and
C
2
1.8.MachineLearning
241
OpenCV-PythonTutorialsDocumentation,Release1
Step:2
-Itcalculatesthedistancefromeachpointtobothcentroids.Ifatestdataismorecloserto
C
1
,thenthatdata
islabelledwith`0'.Ifitiscloserto
C
2
Step:3
-Nextwecalculatetheaverageofallbluepointsandredpointsseparatelyandthatwillbeournewcentroids.
Thatis
C
1
and
C
2
shifttonewlycalculatedcentroids.(Remember,theimagesshownarenottruevaluesandnotto
truescale,itisjustfordemonstrationonly).
Andagain,performstep2withnewcentroidsandlabeldatato`0'and`1'.
242Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Now
Step-2
and
Step-3
areiterateduntilbothcentroidsareconvergedtoxedpoints.
(Oritmaybestopped
1.8.MachineLearning
243
OpenCV-PythonTutorialsDocumentation,Release1
244Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
2.
nclusters(K)
:Numberofclustersrequiredatend
3.
criteria
[Itistheiterationterminationcriteria.Whenthiscriteriaissatised,algorithmiterationstops.Actually,
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
x
=
np
.
random
.
randint(
25
,
100
,
25
)
y
=
np
.
random
.
randint(
175
,
255
,
25
)
z
=
np
.
hstack((x,y))
z
=
z
.
reshape((
50
,
1
))
z
=
np
.
float32(z)
plt
.
hist(z,
256
,[
0
,
256
]),plt
.
show()
Sowehave`z'whichisanarrayofsize50,andvaluesrangingfrom0to255.Ihavereshaped`z'toacolumnvector.
Itwillbemoreusefulwhenmorethanonefeaturesarepresent.ThenImadedataofnp.oat32type.
1.8.MachineLearning
245
OpenCV-PythonTutorialsDocumentation,Release1
NowweapplytheKMeansfunction.Beforethatweneedtospecifythe
criteria
.Mycriteriaissuchthat,whenever
10iterationsofalgorithmisran,oranaccuracyof
epsilon=1.0
#Definecriteria=(type,max_iter=10,epsilon=1.0)
criteria
=
(cv2
.
TERM_CRITERIA_EPS
+
cv2
.
TERM_CRITERIA_MAX_ITER,
10
,
1.0
)
Thisgivesusthecompactness,labelsandcenters.Inthiscase,Igotcentersas60and207.Labelswillhavethesame
A
=
z[labels
==
0
]
B
=
z[labels
==
1
]
NowweplotAinRedcolorandBinBluecolorandtheircentroidsinYellowcolor.
#NowplotAinred,Binblue,centersinyellow
plt
.
hist(A,
256
,[
0
,
256
],color
=

r

)
plt
.
hist(B,
256
,[
0
,
256
],color
=

b

)
plt
.
hist(centers,
32
,[
0
,
256
],color
=

y

)
plt
.
show()
246Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Belowistheoutputwegot:
2.DatawithMultipleFeatures
Inpreviousexample,wetookonlyheightfort-shirtproblem.Here,wewilltakebothheightandweight,ietwo
features.
Remember,inpreviouscase,wemadeourdatatoasinglecolumnvector.Eachfeatureisarrangedinacolumn,while
eachrowcorrespondstoaninputtestsample.
1.8.MachineLearning
247
OpenCV-PythonTutorialsDocumentation,Release1
NowIamdirectlymovingtothecode:
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
X
=
np
.
random
.
randint(
25
,
50
,(
25
,
2
))
Y
=
np
.
random
.
randint(
60
,
85
,(
25
,
2
))
Z
=
np
.
vstack((X,Y))
#converttonp.float32
Z
=
np
.
float32(Z)
#definecriteriaandapplykmeans()
criteria
=
(cv2
.
TERM_CRITERIA_EPS
+
cv2
.
TERM_CRITERIA_MAX_ITER,
10
,
1.0
)
248Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
plt
.
show()
3.ColorQuantization
ColorQuantizationistheprocessofreducingnumberofcolorsinanimage.Onereasontodosoistoreducethe
import
numpy
as
np
import
cv2
img
=
cv2
.
imread(

home.jpg

)
Z
=
img
.
reshape((
-
1
,
3
))
#converttonp.float32
Z
=
np
.
float32(Z)
#definecriteria,numberofclusters(K)andapplykmeans()
criteria
=
(cv2
.
TERM_CRITERIA_EPS
+
cv2
.
TERM_CRITERIA_MAX_ITER,
10
,
1.0
)
K
=
8
1.8.MachineLearning
249
OpenCV-PythonTutorialsDocumentation,Release1
SeetheresultbelowforK=8:
AdditionalResources
Exercises
ComputationalPhotography
250Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
SeeagoodtechniquetoremovenoisesinimagescalledNon-LocalMeans
Denoising

ImageInpainting
Doyouhaveaolddegradedphotowithmanyblackspotsandstrokesonit?
1.9.ComputationalPhotography251
OpenCV-PythonTutorialsDocumentation,Release1
ImageDenoising
Goal
Inthischapter,

YouwilllearnaboutNon-localMeansDenoisingalgorithmtoremovenoiseintheimage.

Youwillseedifferentfunctionslike
cv2.fastNlMeansDenoising()
,
cv2.fastNlMeansDenoisingColored()
252Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Thebluepatchesintheimagelooksthesimilar.Greenpatcheslookssimilar.Sowetakeapixel,takesmallwindow
aroundit,searchforsimilarwindowsintheimage,averageallthewindowsandreplacethepixelwiththeresultwe
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
img
=
cv2
.
imread(

die.png

)
dst
=
cv2
.
fastNlMeansDenoisingColored(img,
None
,
10
,
10
,
7
,
21
)
plt
.
subplot(
121
),plt
.
imshow(img)
plt
.
subplot(
122
),plt
.
imshow(dst)
plt
.
show()
Belowisazoomedversionofresult.Myinputimagehasagaussiannoiseof

=25
.Seetheresult:
1.9.ComputationalPhotography253
OpenCV-PythonTutorialsDocumentation,Release1
2.cv2.fastNlMeansDenoisingMulti()
import
numpy
as
np
import
cv2
from
matplotlib
import
pyplot
as
plt
cap
=
cv2
.
VideoCapture(

vtest.avi

)
#createalistoffirst5frames
img
=
[cap
.
read()[
1
]
for
i
in
xrange(
5
)]
#convertalltograyscale
gray
=
[cv2
.
cvtColor(i,cv2
.
COLOR_BGR2GRAY)
for
i
in
img]
254Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
#convertalltofloat64
gray
=
[np
.
float64(i)
for
i
in
gray]
#createanoiseofvariance25
noise
=
np
.
random
.
randn(
*
gray[
1
]
.
shape)
*
10
#Addthisnoisetoimages
noisy
=
[i
+
noise
for
i
in
gray]
#Convertbacktouint8
noisy
=
[np
.
uint8(np
.
clip(i,
0
,
255
))
for
i
in
noisy]
#Denoise3rdframeconsideringallthe5frames
dst
=
cv2
.
fastNlMeansDenoisingMulti(noisy,
2
,
5
,
None
,
4
,
7
,
35
)
plt
.
subplot(
131
),plt
.
imshow(gray[
2
],

gray

)
plt
.
subplot(
132
),plt
.
imshow(noisy[
2
],

gray

)
plt
.
subplot(
133
),plt
.
imshow(dst,

gray

)
plt
.
show()
Belowimageshowsazoomedversionoftheresultwegot:
1.9.ComputationalPhotography255
OpenCV-PythonTutorialsDocumentation,Release1
256Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Ittakesconsiderableamountoftimeforcomputation.Intheresult,rstimageistheoriginalframe,secondisthe
noisyone,thirdisthedenoisedimage.
AdditionalResources
1.
http://www.ipol.im/pub/art/2011/bcm_nlm/
SeveralalgorithmsweredesignedforthispurposeandOpenCVprovidestwoofthem.Bothcanbeaccessedbythe
samefunction,
cv2.inpaint()
Firstalgorithmisbasedonthepaper
1.9.ComputationalPhotography257
OpenCV-PythonTutorialsDocumentation,Release1
ensuresthosepixelsneartheknownpixelsareinpaintedrst,sothatitjustworkslikeamanualheuristicoperation.
Thisalgorithmisenabledbyusingtheag,
cv2.INPAINT_TELEA
.
Secondalgorithmisbasedonthepaper
“Navier-Stokes,FluidDynamics,andImageandVideoInpainting”
by
Bertalmio,Marcelo,AndreaL.Bertozzi,andGuillermoSapiroin2001.Thisalgorithmisbasedonuiddynamicsand
utilizespartialdifferentialequations.Basicprincipleisheurisitic.Itrsttravelsalongtheedgesfromknownregions
tounknownregions(becauseedgesaremeanttobecontinuous).Itcontinuesisophotes(linesjoiningpointswithsame
intensity,justlikecontoursjoinspointswithsameelevation)whilematchinggradientvectorsattheboundaryofthe
reduceminimumvarianceinthatarea.Thisalgorithmisenabledbyusingtheag,
cv2.INPAINT_NS
.
Code
Weneedtocreateamaskofsamesizeasthatofinputimage,wherenon-zeropixelscorrespondstotheareawhich
istobeinpainted.Everythingelseissimple.Myimageisdegradedwithsomeblackstrokes(Iaddedmanually).I
createdacorrespondingstrokeswithPainttool.
import
numpy
as
np
import
cv2
img
=
cv2
.
imread(

messi_2.jpg

)
mask
=
cv2
.
imread(

mask2.png

,
0
)
dst
=
cv2
.
inpaint(img,mask,
3
,cv2
.
INPAINT_TELEA)
cv2
.
imshow(

dst

,dst)
cv2
.
waitKey(
0
)
cv2
.
destroyAllWindows()
Seetheresultbelow.Firstimageshowsdegradedinput.Secondimageisthemask.Thirdimageistheresultofrst
algorithmandlastimageistheresultofsecondalgorithm.
258Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
Bertalmio,Marcelo,AndreaL.Bertozzi,andGuillermoSapiro.“Navier-stokes,uiddynamics,andimageand
videoinpainting.”InComputerVisionandPatternRecognition,2001.CVPR2001.Proceedingsofthe2001
OpenCV-PythonTutorialsDocumentation,Release1
260Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
Nowallpossiblesizesandlocationsofeachkernelisusedtocalculateplentyoffeatures.(Justimaginehowmuch
computationitneeds?Evena24x24windowresultsover160000features).Foreachfeaturecalculation,weneedto
ndsumofpixelsunderwhiteandblackrectangles.Tosolvethis,theyintroducedtheintegralimages.Itsimplies
calculationofsumofpixels,howlargemaybethenumberofpixels,toanoperationinvolvingjustfourpixels.Nice,
isn'tit?Itmakesthingssuper-fast.
Butamongallthesefeatureswecalculated,mostofthemareirrelevant.Forexample,considertheimagebelow.Top
rowshowstwogoodfeatures.Therstfeatureselectedseemstofocusonthepropertythattheregionoftheeyesis
oftendarkerthantheregionofthenoseandcheeks.Thesecondfeatureselectedreliesonthepropertythattheeyes
OpenCV-PythonTutorialsDocumentation,Release1
aredarkerthanthebridgeofthenose.Butthesamewindowsapplyingoncheeksoranyotherplaceisirrelevant.So
howdoweselectthebestfeaturesoutof160000+features?Itisachievedby
Adaboost
.
Forthis,weapplyeachandeveryfeatureonallthetrainingimages.Foreachfeature,itndsthebestthresholdwhich
willclassifythefacestopositiveandnegative.Butobviously,therewillbeerrorsormisclassications.Weselectthe
featureswithminimumerrorrate,whichmeanstheyarethefeaturesthatbestclassiesthefaceandnon-faceimages.
(Theprocessisnotassimpleasthis.Eachimageisgivenanequalweightinthebeginning.Aftereachclassication,
weightsofmisclassiedimagesareincreased.Thenagainsameprocessisdone.Newerrorratesarecalculated.Also
newweights.Theprocessiscontinueduntilrequiredaccuracyorerrorrateisachievedorrequirednumberoffeatures
arefound).
Finalclassierisaweightedsumoftheseweakclassiers.Itiscalledweakbecauseitalonecan'tclassifytheimage,
10featuresoutof6000+areevaluatedpersub-window.
262Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
import
numpy
as
np
import
cv2
face_cascade
=
cv2
.
CascadeClassifier(

haarcascade_frontalface_default.xml

)
eye_cascade
=
cv2
.
CascadeClassifier(

haarcascade_eye.xml

)
img
=
cv2
.
imread(

sachin.jpg

)
gray
=
cv2
.
cvtColor(img,cv2
.
COLOR_BGR2GRAY)
faces
=
face_cascade
.
Resultlookslikebelow:
OpenCV-PythonTutorialsDocumentation,Release1
AdditionalResources
1.
VideoLectureon
LearnhowOpenCV-Pythonbindingsaregenerated.
264Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
HowOpenCV-PythonBindingsWorks?
Goal
Learn:

HowOpenCV-Pythonbindingsaregenerated?

HowtoextendnewOpenCVmodulestoPython?
HowOpenCV-Pythonbindingsaregenerated?
InOpenCV,allalgorithmsareimplementedinC++.Butthesealgorithmscanbeusedfromdifferentlanguageslike
1.11.OpenCV-PythonBindings265
OpenCV-PythonTutorialsDocumentation,Release1
HowtoextendnewmodulestoPython?
Headerparserparsetheheaderlesbasedonsomewrappermacrosaddedtofunctiondeclaration.Enumeration
CV_EXPORTS_W
void
equalizeHist
(InputArraysrc,OutputArraydst);
Headerparsercanunderstandtheinputandoutputargumentsfromkeywordslike
InputArray,OutputArray
CV_EXPORTS_W
void
minEnclosingCircle
(InputArraypoints,
CV_OUTPoint2f
&
center,CV_OUT
float
&
radius);
Forlargeclassesalso,
CV_EXPORTS_W
class
CV_EXPORTS_W
CLAHE
:
public
Algorithm
{
public
:
CV_WRAP
virtual
void
apply(InputArraysrc,OutputArraydst)
=
0
;
CV_WRAP
virtual
void
Overloadedfunctionscanbeextendedusing
CV_EXPORTS_AS
.Butweneedtopassanewnamesothateachfunction
willbecalledbythatnameinPython.Takethecaseofintegralfunctionbelow.Threefunctionsareavailable,soeach
oneisnamedwithasufxinPython.Similarly
CV_WRAP_AS
//!computestheintegralimage
CV_EXPORTS_W
void
integral
(InputArraysrc,OutputArraysum,
int
sdepth
=
-
1
);
//!computestheintegralimageandintegralforthesquaredimage
CV_EXPORTS_AS(integral2)
void
integral(InputArraysrc,OutputArraysum,
OutputArraysqsum,
int
sdepth
=
-
1
,
int
,
!
sqdepth
=
-
1
);
//!computestheintegralimage,integralforthesquaredimageandthetilted
,
!
integralimage
CV_EXPORTS_AS(integral3)
void
integral(InputArraysrc,OutputArraysum,
OutputArraysqsum,OutputArraytilted,
int
sdepth
=
-
1
,
int
sqdepth
=
-
1
);
Smallclasses/structsareextendedusing
CV_EXPORTS_W_SIMPLE
.ThesestructsarepassedbyvaluetoC++func-
class
CV_EXPORTS_W_SIMPLE
DMatch
{
public
:
CV_WRAPDMatch();
CV_WRAP
DMatch
(
int
_queryIdx,
int
_trainIdx,
float
_distance);
CV_WRAP
DMatch
(
int
_queryIdx,
int
_trainIdx,
int
_imgIdx,
float
_distance);
266Chapter1.OpenCV-PythonTutorials
OpenCV-PythonTutorialsDocumentation,Release1
CV_PROP_RW
int
queryIdx;
//querydescriptorindex
CV_PROP_RW
int
trainIdx;
//traindescriptorindex
CV_PROP_RW
int
imgIdx;
//trainimageindex
CV_PROP_RW
float
distance;
};
Someothersmallclasses/structscanbeexportedusing
CV_EXPORTS_W_MAP
whereitisexportedtoaPythonnative
dictionary.Moments()isanexampleofit.
class
CV_EXPORTS_W_MAP
Moments
{
public
:
//!spatialmoments
CV_PROP_RW
double
m00,m10,m01,m20,m11,m02,m30,m21,m12,m03;
//!centralmoments
CV_PROP_RW
double
mu20,mu11,mu02,mu30,mu21,mu12,mu03;
//!centralnormalizedmoments
CV_PROP_RW
double
nu20,nu11,nu02,nu30,nu21,nu12,nu03;
};
SothesearethemajorextensionmacrosavailableinOpenCV.Typically,adeveloperhastoputpropermacrosin
1.11.OpenCV-PythonBindings267
OpenCV-PythonTutorialsDocumentation,Release1
268Chapter1.OpenCV-PythonTutorials
CHAPTER
2
Indicesandtables

genindex

modindex

search
269

Приложенные файлы

  • pdf 11179767
    Размер файла: 5 MB Загрузок: 0

Добавить комментарий