3

Is there any way to run "describe extended table" in Hive and have the results returned in JSON, XML, or some easily parsed format? Or is there an existing parse in Java that can parse this type of format?

Table(tableName:test3, dbName:testdatabase, owner:johnsmith, createTime:1481135997, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:a, type:string, comment:""a,m)], location:hdfs://testcluster/apps/hive/warehouse/testdatabase/test3, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, numFiles=0, transient_lastDdlTime=1481135997}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)

Hari
  • 192
  • 2
  • 12
leontp587
  • 791
  • 2
  • 9
  • 21
  • For humans, SHOW CREATE TABLE is way more readable *(IMHHO - in my humble human opinion)* – Samson Scharfrichter Dec 08 '16 at 13:40
  • 1
    And if you really want to access the raw information in Java, try the Metastore API -- poorly documented but taps into the real thing http://hive.apache.org/javadocs/r1.1.1/api/org/apache/hadoop/hive/metastore/api/package-summary.html *(adjust to the actual Hive version you are using)* – Samson Scharfrichter Dec 08 '16 at 13:44
  • Another option is to access the Metastore (!) that is also SQL... But [it is possible with beeline?](https://stackoverflow.com/q/57401967/287948) – Peter Krauss Aug 07 '19 at 20:46

2 Answers2

3

Maybe you can use hive.ddl.output.format to format output

set hive.ddl.output.format=json;
desc extended table;
saboloh
  • 31
  • 5
  • Not works, *"Error: Error while processing statement: Cannot modify hive.ddl.output.format at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)"* – Peter Krauss Aug 07 '19 at 20:44
  • which hive version? [hive config source code](https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java) can find this config, maybe your hive server can't change some params when runtime, you can check your hive config this settings: `set hive.conf.restricted.list;` [hive config details](https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties) – saboloh Aug 09 '19 at 01:28
  • Hi Saboloh, thanks the clues! I am using Hive *v1.2*... Is difficult to change corporative cluster... My local `hive.conf.restricted.list` is *hive.security.authenticator.manager, hive.security.authorization.manager, hive.security.metastore.authorization.manager, hive.security.metastore.authenticator.manager, hive.users.in.admin.role,hive.server2.xsrf.filter.enabled, hive.security.authorization.enabled* – Peter Krauss Aug 09 '19 at 13:04
2

DESCRIBE FORMATTED my_table; is more friendly
not json or xml.. but more readable than DESCRIBE EXTENDED my_table;

belostoky
  • 934
  • 2
  • 11
  • 22
  • 1
    It is not structured data, **is not friendly for programemers, data filters, etc.**. Need JSON or XML, like in any other good SGDB (like PostgreSQL or Oracle). – Peter Krauss Aug 07 '19 at 20:48