Transfer Data from CSV files to Deta Bases + Alternative solution.

Transfer Data from CSV files to Deta Bases + Alternative solution.

Subscribe to my newsletter and never miss my upcoming articles

What are we going to do.

First of all , lets go through this in two phases.

  • Convert csv to json
  • Upload converted json to Deta base

Prerequisites

  • Create a free account in Deta
  • Create a new project (Note down the project key, we need it)

Lets start.

Imagine having a csv file as

name,age
fredy,34
god,34345345
mannu,34

We want it like:

[{'name': 'fredy', 'age': '34'},{'name': 'god', 'age': '34345345'}
,{'name': 'maanu', 'age': '34'}
]

Lets start by importing csv module

import csv

Read the file contents:

arr=[]
with open (path) as csvFile:
    csvReader = csv.DictReader(csvFile)
    for csvRow in csvReader:
        arr.append(csvRow)
  • We are making a list named arr for appending all the json extracted from csv.
csvReader = csv.DictReader(csvFile)
  • This creates a csvreader object which maps each row to a dictionary.

  • We can now use the csvreader to extract the dictionary.

for csvRow in csvReader:
      arr.append(csvRow)
  • The above loop takes each dictionary from csvreader object and append it to list arr,which i mentioned earlier.

  • We can now print the list arr to see the json data.

>> print(arr)
[{'name': 'fredy', 'age': '34'},{'name': 'god', 'age': '34345345'}
,{'name': 'maanu', 'age': '34'}
]

Csv file is converted , lets upload it to Deta Base

  • Install deta module.
    pip install deta
    
  • Lets import it.
    from deta import Deta
    
deta = Deta(projectkey)
database = deta.Base(db)
for each in arr:
    datafile.insert(each)
    print("Uploaded {}".format(each))
  • Initialize with a Project Key( which we get while creating a project)
database = deta.Base(db)
  • Connect or create a database in Deta
for each in arr:
    datafile.insert(each)
    print("Uploaded {}".format(each))
  • We are iterating through the list arr and datafile.insert(each) inserts each data into the database.

  • Finally we have this in Deta Base UI.

deta

Final Code

def csvtodeta(id,path,db):
    deta = Deta(id)
    datafile = deta.Base(db)
    print("Reading data from {}".format(path))
    arr=[]
    with open (path) as csvFile:
        csvReader = csv.DictReader(csvFile)
        for csvRow in csvReader:
           arr.append(csvRow)
    for each in arr:
        datafile.insert(each)
        print("Uploaded {}".format(each))
    print("succesfully uploaded {} data to {} Base".format(len(arr),db))

Alternative Solution

I created a python CLI tool to do the same.

pip install csvtodeta
  • With just one command the data of csv file is uploaded to DetaπŸŽ‰.
$ csvtodeta --id yourprojectkey --path path/to/csv.csv --db detabasename
  • id is your project key from deta
  • Provide the path name of csv file.
  • Provide database name of the Base you want to create or connect.
  • Tada...πŸŽ‰πŸŽ‰ The data is uploaded.
$ csvtodeta --id 45345dhsgh3rjdf2ur34hhwf --path src/deta.csv --db detabasename
Reading data from src/deta.csv
Uploaded {'name': 'fredy', 'age': '34'}
Uploaded {'name': 'god', 'age': '34345345'}
Uploaded {'name': 'maanu', 'age': '34'}
succesfully uploaded 3 data to detabasename Base

Github Repo

Thank you for reading this blog. If you have any doubts or if there is any wrong explanation please comment down here.

#python3#data-science#databases
Β 
Share this