donderdag 5 maart 2020

miniconda anaconda configuratie

-hoe activeer je environment in een Anaconda prompt

activate  O:\Input\03_ZichtenGrip\python\environments\ENV_flask_dash

let op UNC  paden

\\swappams2820.basis.lan\IVS_BI_Ontwikkel\data\Input\03_ZichtenGrip\python\environments\ENV_flask_dash



- Hoe wordt anaconda prompt aangeroepen

call "%windir%\System32\cmd.exe "/K" C:\Users\wagene002\AppData\Local\Continuum\miniconda3\Scripts\activate.bat C:\Users\wagene002\AppData\Local\Continuum\miniconda3"

- Hoe wordt anaconda prompt aangeroepen met eigen Environment (startConda Environment.bat)

echo "hi"
call "%windir%\System32\cmd.exe "/K"  C:\Users\wagene002\AppData\Local\Continuum\miniconda3\Scripts\activate.bat O:\Input\03_ZichtenGrip\python\environments\ENV_flask_dash\
C:\Users\wagene002\Desktop\runit.bat
echo "bye"


For Windows, use the following script in your batch file to execute a Python script. Simply change your personal file paths.

cmd /c C:\ProgramData\Anaconda3\condabin\conda.bat run "C:\ProgramData\Anaconda3\python.exe" "C:\Users\User Name\Path to your Python File\Python File.py"


hoe start je een python script

cmd /c C:\Users\wagene002\AppData\Local\Continuum\miniconda3\condabin\conda.bat run "\\swappams2820.basis.lan\IVS_BI_Ontwikkel\data\Input\03_ZichtenGrip\python\environments\ENV_flask_dash\python.exe" "\\swappams2820.basis.lan\IVS_BI_Ontwikkel\data\Input\03_ZichtenGrip\python\scripts\CheckStagingWMON.py"



hoe start je de juiste miniconda omgeving en geef je daarna het juiste python commanda

call "%windir%\System32\cmd.exe "/K"  C:\Users\wagene002\AppData\Local\Continuum\miniconda3\Scripts\activate.bat O:\Input\03_ZichtenGrip\python\environments\ENV_flask_dash\  & cmd /c C:\Users\wagene002\AppData\Local\Continuum\miniconda3\condabin\conda.bat run "\\swappams2820.basis.lan\IVS_BI_Ontwikkel\data\Input\03_ZichtenGrip\python\environments\ENV_flask_dash\python.exe" "\\swappams2820.basis.lan\IVS_BI_Ontwikkel\data\Input\03_ZichtenGrip\python\scripts\CheckStagingWMON.py"

let op de essentie zit hem in eerst de shel starten van miniconda en daarna na de & het commando geven


conda offline installation

conda install <package-file-name>.tar.bz2

 C:\Users\wagene002\Desktop>conda install O:\Input\03_ZichtenGrip\pandas-datareader-0.8.1-py_0.tar.bz2

woensdag 4 maart 2020

Vergelijken dataframes

Vergelijken van 2 dataframes met elkaaar


def VergelijkTabellen(dfBase,dfCurrent) :
    dfC=pd.merge(dfBase,dfCurrent, on='table_name', how='outer',suffixes=('_old', '_new'))
#    Analyses
    logger.info('------------------------------' + str(datetime.now()) + '------------------------------' )
  
    logger.info('tabellen alleen in old' )
    t=dfC[(dfC['aantal_old'].isna() &  dfC['aantal_new'].notna())]
    logger.info(t.to_string(columns=['table_name'],index=False))
    logger.info('------------------------------')
    logger.info('tabellen alleen in new' )
    t=dfC[(dfC['aantal_old'].notna() &  dfC['aantal_new'].isna())]
    logger.info(t.to_string(columns=['table_name'],index=False))
    t=dfC[(dfC['aantal_old'].notna() &  dfC['aantal_new'].notna())]
    dfC['Verschil']=dfC.apply(lambda x : x['aantal_new'] - x['aantal_old'], axis=1)
  
    lstValues=[(dfC['aantal_new'] - dfC['aantal_old']) /(dfC['aantal_old'])]
    lstConditions = [dfC['aantal_old'].notna() & dfC['aantal_new'].notna() & dfC['aantal_old']!=0 ]
    dfC['Stijging']= np.select(lstConditions,lstValues,'nvt')
    dfC['Stijging'] = dfC['Stijging'].map(lambda x : '{percent:.2%}'.format(percent=float(x)) if x!= 'nvt' else x)
  
    logger.info(dfC.to_string(index=False))
    return dfC

Percentages printen van strings.


    dfC['Stijging'] = dfC['Stijging'].map(lambda x : '{percent:.2%}'.format(percent=float(x)))


Percentage berekenen


     lstValues=[(dfC['aantal_new'] - dfC['aantal_old']) /(dfC['aantal_old'])]
    lstConditions = [dfC['aantal_old'].notna() & dfC['aantal_new'].notna() & dfC['aantal_old']!=0 ]
    dfC['Stijging']= np.select(lstConditions,lstValues,'nvt')

    dfC['Stijging'] = dfC['Stijging'].map(lambda x : '{percent:.2%}'.format(percent=float(x)) if x!= 'nvt' else x)

dinsdag 3 maart 2020

Create pandas columns based on multiple conditions

Create columns based on other values in the row


 df1['Verschil']=df1.apply(lambda x : x['aantal_new'] - x['aantal_old'], axis=1)


Create pandas columns based on multiple conditions


With pandas and numpy we barely have to write our own functions, especially since our own functions will perform slow because these are not vectorized and pandas + numpy provide a rich pool of vectorized methods for us.

In this case your are looking for np.select since you want to create a column based on multiple conditions:

definieer 2 lijsten van dezelfde lengte
1 lijst met de where clauses
1 lijst met de values


lstValues=[(df1['aantal_new'] - df1['aantal_old']) /(df1['aantal_old']),'99999999999']

lstConditions = [df1['aantal_old'].notna() & df1['aantal_new'].notna(), df1['aantal_old'].isna() &  

df1['aantal_new'].isna()]
 

df1['Stijging2']= np.select(lstConditions,lstValues,'99')

Datums bepalen adhv begin en einddatum in Dataframe

Voorbeeld op losse velden  ####################################################################### # import necessary packages from datetime...