Cara menggunakan json select python

Your JSON contains a list, with one dictionary in it; there are two square brackets, [ and ], around the dictionary.

Select the first element:

print(data[0]['students'])

Quick demo:

>>> print(data)
[{'instructor': 'hamarneh', 'course': 'CMPT 102 D1', 'title': 'Scientific Cmpt.Prgm', 'students': ['axc5', 'csf10', 'ctu1', 'nmw15', 'nsm12', 'ppy1', 'qtg13', 'tim1', 'tkd10', 'vhm8', 'vsv1', 'wps1', 'xup12', 'yqt6']}]
>>> print(data[0]['students'])
['axc5', 'csf10', 'ctu1', 'nmw15', 'nsm12', 'ppy1', 'qtg13', 'tim1', 'tkd10', 'vhm8', 'vsv1', 'wps1', 'xup12', 'yqt6']

Note that you could have spotted this yourself with a quick print of just data.

If this was a list of multiple courses and you need to count per-student, set up a dictionary keyed on students, containing integers. Using collections.defaultdict() makes that a little easier:

from collections import defaultdict

courses_per_student = defaultdict(int)

for course in data:
    for student in course['students']:
        courses_per_student[student] += 1

From the the link you pasted to pastebin and from the json that I saw, its a you issues as list containing key, fields(which holds custom fields), self, id, expand.

You can simply iterate through this response and extract values for keys you want. You can go like.

data = response.json()
issues = data.get('issues', list())

x = list()

for issue in issues:
    temp = {
        'key': issue['key'],
        'customfield': issue['fields']['customfield_12830'],
        'total': issue['fields']['progress']['total']
    }
    x.append(temp)
print(x)

x is list of dictionaries containing the data for fields you mentioned. Let me know if I have been unclear somewhere or what I have given is not what you are looking for.

PS: It is always advisable to use dict.get('keyname', None) to get values as you can always put a default value if key is not found. For this solution I didn't do it as I just wanted to provide approach.

Update: In the comments you(OP) mentioned that it gives attributerror.Try this code

data = response.json()
issues = data.get('issues', list())

x = list()

for issue in issues:
    temp = dict()
    key = issue.get('key', None)
    if key:
       temp['key'] = key 

    fields = issue.get('fields', None)
    if fields:
        customfield = fields.get('customfield_12830', None)
        temp['customfield'] = customfield

        progress = fields.get('progress', None)
        if progress:
            total = progress.get('total', None)
            temp['total'] = total

    x.append(temp)
print(x)

cara menerapkan WHERE klausa pada kolom JSON untuk melakukan kueri SELECT pada tabel yang memiliki dua kolom (id Integer, attr JSON). JSON bersarang dan dalam kondisi filter hanya ada satu pasangan nilai kunci json yang dibolehkan. Pasangan nilai kunci ini bisa di mana saja di Josn.

+----+-----------------------------------------------------------------
| id | attr                                                                                          
|
+----+-----------------------------------------------------------------
|  1 | {"id":"0001","type":"donut","name":"Cake","ppu":0.55}                                         
|
|  2 | {"id":"0002","type":"donut","name":"Cake","ppu":0.55,"batters":
       {"batter1":100,"batter2":200}} 
+----+-----------------------------------------------------------------

Lompati ke konten utama

Browser ini sudah tidak didukung.

Mutakhirkan ke Microsoft Edge untuk memanfaatkan fitur, pembaruan keamanan, dan dukungan teknis terkini.

Membuat DataFrame dari string JSON atau kamus Python

• Artikel
• 09/22/2022
• 2 menit untuk membaca

Dalam artikel ini

Pada artikel ini kita akan meninjau bagaimana Anda dapat membuat Apache Spark DataFrame dari variabel yang berisi untai (karakter) JSON atau kamus Python.

Membuat Spark DataFrame dari untai (karakter) JSON

1. Tambahkan konten JSON dari variabel ke daftar.

   import scala.collection.mutable.ListBuffer
   val json_content1 = "{'json_col1': 'hello', 'json_col2': 32}"
   val json_content2 = "{'json_col1': 'hello', 'json_col2': 'world'}"
   
   var json_seq = new ListBuffer[String]()
   json_seq += json_content1
   json_seq += json_content2
   

2. Buat himpunan data Spark dari daftar.

   val json_ds = json_seq.toDS()
   

3. Gunakan spark.read.json untuk menguraikan himpunan data Spark.

   val df= spark.read.json(json_ds)
   display(df)
   

Kode sampel gabungan

Blok kode sampel ini menggabungkan langkah-langkah sebelumnya menjadi contoh individual. Sampel Python dan Scala melakukan tugas yang sama.

json_content1 = "{'json_col1': 'hello', 'json_col2': 32}"
json_content2 = "{'json_col1': 'hello', 'json_col2': 'world'}"

json_list = []
json_list.append(json_content1)
json_list.append(json_content2)

df = spark.read.json(sc.parallelize(json_list))
display(df)

import scala.collection.mutable.ListBuffer
val json_content1 = "{'json_col1': 'hello', 'json_col2': 32}"
val json_content2 = "{'json_col1': 'hello', 'json_col2': 'world'}"

var json_seq = new ListBuffer[String]()
json_seq += json_content1
json_seq += json_content2

val json_ds = json_seq.toDS()
val df= spark.read.json(json_ds)
display(df)

Mengekstrak kolom untai (karakter) dengan data JSON dari DataFrame dan mengurainya

1. Pilih kolom JSON dari DataFrame dan konversi ke RDD jenis RDD[Row].

   import org.apache.spark.sql.functions._
   
   val test_df = Seq(("1", "{'json_col1': 'hello', 'json_col2': 32}", "1.0"),("1", "{'json_col1': 'hello', 'json_col2': 'world'}", "1.0")).toDF("row_number", "json", "token")
   
   val row_rdd = test_df.select(col("json")).rdd  // Selecting just the JSON column and converting it to RDD.
   

2. Konversi RDD[Row] ke RDD[String].

   val string_rdd = row_rdd.map(_.mkString(","))
   

3. Gunakan spark.read.json untuk menguraikan RDD[String].

   val df1= spark.read.json(string_rdd)
    display(df1)
   

Kode sampel gabungan

Blok kode sampel ini menggabungkan langkah-langkah sebelumnya menjadi satu contoh.

import org.apache.spark.sql.functions._

val test_df = Seq(("1", "{'json_col1': 'hello', 'json_col2': 32}", "1.0"),("1", "{'json_col1': 'hello', 'json_col2': 'world'}", "1.0")).toDF("row_number", "json", "token")

val row_rdd = test_df.select(col("json")).rdd
val string_rdd = row_rdd.map(_.mkString(","))

val df1= spark.read.json(string_rdd)
display(df1)

Membuat Spark DataFrame dari direktori Python

1. Periksa jenis data dan konfirmasikan bahwa itu adalah jenis kamus.

   jsonDataDict = {"job_id":33100,"run_id":1048560,"number_in_job":1,"state":{"life_cycle_state":"PENDING","state_message":"Waiting for cluster"},"task":{"notebook_task":{"notebook_path":"/Users//path/test_notebook"}},"cluster_spec":{"new_cluster":{"spark_version":"4.3.x-scala2.11","attributes":{"type":"fixed_node","memory":"8g"},"enable_elastic_disk":"false","num_workers":1}},"cluster_instance":{"cluster_id":"0000-000000-wares10"},"start_time":1584689872601,"setup_duration":0,"execution_duration":0,"cleanup_duration":0,"creator_user_name":"","run_name":"my test job","run_page_url":"https://testurl.databricks.com#job/33100/run/1","run_type":"SUBMIT_RUN"}
   
   type(jsonDataDict)
   

2. Gunakan json.dumps untuk mengonversi kamus Python menjadi untai (karakter) JSON.

   import json
   jsonData = json.dumps(jsonDataDict)
   

3. Tambahkan isi JSON ke daftar.

   jsonDataList = []
   jsonDataList.append(jsonData)
   

4. Konversi daftar ke RDD dan uraikan menggunakan spark.read.json.

   jsonRDD = sc.parallelize(jsonDataList)
   df = spark.read.json(jsonRDD)
   display(df)
   

Kode sampel gabungan

Blok kode sampel ini menggabungkan langkah-langkah sebelumnya menjadi satu contoh.

jsonDataDict = {"job_id":33100,"run_id":1048560,"number_in_job":1,"state":{"life_cycle_state":"PENDING","state_message":"Waiting for cluster"},"task":{"notebook_task":{"notebook_path":"/Users//path/test_notebook"}},"cluster_spec":{"new_cluster":{"spark_version":"4.3.x-scala2.11","attributes":{"type":"fixed_node","memory":"8g"},"enable_elastic_disk":"false","num_workers":1}},"cluster_instance":{"cluster_id":"0000-000000-wares10"},"start_time":1584689872601,"setup_duration":0,"execution_duration":0,"cleanup_duration":0,"creator_user_name":"","run_name":"my test job","run_page_url":"https://testurl.databricks.com#job/33100/run/1","run_type":"SUBMIT_RUN"}

type(jsonDataDict)

import json
jsonData = json.dumps(jsonDataDict)

jsonDataList = []
jsonDataList.append(jsonData)

jsonRDD = sc.parallelize(jsonDataList)
df = spark.read.json(jsonRDD)
display(df)

Contoh buku catatan

Mengurai contoh untai (karakter) JSON atau kamus Python

Mendapatkan buku catatan

Kapan JSON digunakan?

Apa itu JSON di Python?

Apa 2 struktur pembentuk JSON?

Apa itu JSON dan contohnya?