Transfer Data from CSV files to Deta Bases + Alternative solution.

Subscribe to my newsletter and never miss my upcoming articles

What are we going to do.

First of all , lets go through this in two phases.

  • Convert csv to json
  • Upload converted json to Deta base

Prerequisites

  • Create a free account in Deta
  • Create a new project (Note down the project key, we need it)

Lets start.

Imagine having a csv file as

name,age
fredy,34
god,34345345
mannu,34

We want it like:

[{'name': 'fredy', 'age': '34'},{'name': 'god', 'age': '34345345'}
,{'name': 'maanu', 'age': '34'}
]

Lets start by importing csv module

import csv

Read the file contents:

arr=[]
with open (path) as csvFile:
    csvReader = csv.DictReader(csvFile)
    for csvRow in csvReader:
        arr.append(csvRow)
  • We are making a list named arr for appending all the json extracted from csv.
csvReader = csv.DictReader(csvFile)
  • This creates a csvreader object which maps each row to a dictionary.

  • We can now use the csvreader to extract the dictionary.

for csvRow in csvReader:
      arr.append(csvRow)
  • The above loop takes each dictionary from csvreader object and append it to list arr,which i mentioned earlier.

  • We can now print the list arr to see the json data.

>> print(arr)
[{'name': 'fredy', 'age': '34'},{'name': 'god', 'age': '34345345'}
,{'name': 'maanu', 'age': '34'}
]

Csv file is converted , lets upload it to Deta Base

  • Install deta module.
    pip install deta
    
  • Lets import it.
    from deta import Deta
    
deta = Deta(projectkey)
database = deta.Base(db)
for each in arr:
    datafile.insert(each)
    print("Uploaded {}".format(each))
  • Initialize with a Project Key( which we get while creating a project)
database = deta.Base(db)
  • Connect or create a database in Deta
for each in arr:
    datafile.insert(each)
    print("Uploaded {}".format(each))
  • We are iterating through the list arr and datafile.insert(each) inserts each data into the database.

  • Finally we have this in Deta Base UI.

deta

Final Code

def csvtodeta(id,path,db):
    deta = Deta(id)
    datafile = deta.Base(db)
    print("Reading data from {}".format(path))
    arr=[]
    with open (path) as csvFile:
        csvReader = csv.DictReader(csvFile)
        for csvRow in csvReader:
           arr.append(csvRow)
    for each in arr:
        datafile.insert(each)
        print("Uploaded {}".format(each))
    print("succesfully uploaded {} data to {} Base".format(len(arr),db))

Alternative Solution

I created a python CLI tool to do the same.

pip install csvtodeta
  • With just one command the data of csv file is uploaded to DetaπŸŽ‰.
$ csvtodeta --id yourprojectkey --path path/to/csv.csv --db detabasename
  • id is your project key from deta
  • Provide the path name of csv file.
  • Provide database name of the Base you want to create or connect.
  • Tada...πŸŽ‰πŸŽ‰ The data is uploaded.
$ csvtodeta --id 45345dhsgh3rjdf2ur34hhwf --path src/deta.csv --db detabasename
Reading data from src/deta.csv
Uploaded {'name': 'fredy', 'age': '34'}
Uploaded {'name': 'god', 'age': '34345345'}
Uploaded {'name': 'maanu', 'age': '34'}
succesfully uploaded 3 data to detabasename Base

Github Repo

Thank you for reading this blog. If you have any doubts or if there is any wrong explanation please comment down here.

No Comments Yet