How to identify the output of dex:get as being JSON or HTML in Common Lisp?-CodePudding

I am using SBCL, Emacs, and Slime. Moreover, I am using the library Dexador. Thus, I can do HTTP requests. Some of them will return JSON:

CL-USER> (dex:get "http://ip.jsontest.com/")
 "{\"ip\": \"179.183.248.207\"}

Others will return HTML:

(dex:get "https://ambrevar.xyz/")
"<!DOCTYPE html>
<html lang=\"en\">
<head>
<!-- 2021-12-29 -->
<meta charset=\"utf-8\">
<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">
<title>&lrm;</title>
<meta name=\"generator\" content=\"Org mode\">
<meta name=\"author\" content=\"root\">
<link rel=\"stylesheet\" type=\"text/css\" href=\"../dark.css\">
<link rel=\"icon\" type=\"image/x-icon\" href=\"../logo.png\">
</head>
<body>
<div id=\"content\">
<div id=\"outline-container-org4051da4\" class=\"outline-2\">
<h2 id=\"org4051da4\">Contact</h2>
<div class=\"outline-text-2\" id=\"text-org4051da4\">
<ul class=\"org-ul\">
<li>Email: <a href=\"mailto:[email protected]\">[email protected]</a></li>
<li>PGP: <a href=\"ambrevar.asc\">0x9BDCF497A4BBCC7F</a></li>
</ul>
</div>
</div>
</div>
</body>
</html>
"

After storing the result inside variables with:

CL-USER> (defparameter html-response (dex:get "https://ambrevar.xyz/"))
HTML-RESPONSE

CL-USER> (defparameter json-response (dex:get "http://ip.jsontest.com/"))
JSON-RESPONSE

I would like to create a function to check if the output is JSON or HTML. Hence, I did this function:

(defun html-or-json (response)
  "Check it the server response is HTML or JSON data."
  (cond ((null response) nil)
        ((equal (subseq response 0 1) "<") "html")
        (t "json")))

It works:

CL-USER> (html-or-json json-response)
"json"

CL-USER> (html-or-json html-response)
"html"

However, my solutions seems ugly to me. Initially, I tried using handle-case and serializing the answer as JSON. If it worked, it would be a JSON. If it failed, the function would consider the object as HTML. But things did not go well since I am not proficient with handle-case syntax.

Can you think of an alternative to achieve this goal? Maybe using handle-case?

CodePudding user response：

@ignis volens is right, you need to use the Content-Type header. There is some work to do to update the code.

Using defparameter, part of the Dexador response is discarded

If the function dex:get is called in the REPL, it will return 4 values:

* (dex:get "http://ip.jsontest.com/")
"{\"ip\": \"xx.xx.xx.xxx\"}                       ;; value 1: IP address
"
200                                               ;; value 2: HTTP code
#<HASH-TABLE :TEST EQUAL :COUNT 6 {100A0EA203}>   ;; value 3: HTTP headers
#<QURI.URI.HTTP:URI-HTTP http://ip.jsontest.com/> ;; value 4: URI

When using defparameter, only the first value will be stored in the variable.

* (defparameter response (dex:get "http://ip.jsontest.com/"))
RESPONSE 
* response
"{\"ip\": \"xx.xx.xx.xx\"} ;; value 1: IP address
"
*

One way to save all the values could be to use the function MULTIPLE-VALUE-LIST

* (defparameter response (multiple-value-list (dex:get "http://ip.jsontest.com/")))
RESPONSE
* response
("{\"ip\": \"xx.xx.xx.xx\"}                           ;; value 1: IP address
"
 200                                                  ;; value 2: HTTP code
 #<HASH-TABLE :TEST EQUAL :COUNT 6 {100A146813}>      ;; value 3: HTTP headers
 #<QURI.URI.HTTP:URI-HTTP http://ip.jsontest.com/>)   ;; value 4: URI

Read the Content-Type header from the response

One could write a function to get a field from the HTTP headers, based on the DESTRUCTURING-BIND macro.

(defun get-dex-header (dex-response header)
  (destructuring-bind (raw-response http-code http-headers quri)
      dex-response
    (gethash header http-headers)))

The use of the function is straight-forward:

* (defparameter json-response (multiple-value-list (dex:get "http://ip.jsontest.com/")))
JSON-RESPONSE
* (get-dex-header json-response "content-type")
"application/json"

* (defparameter html-response (multiple-value-list (dex:get "http://bing.com/")))
HTML-RESPONSE
(get-dex-header html-response "content-type")
"text/html; charset=utf-8"

CodePudding user response：

My answer would go also into the direction or @Robert. But I would use multiple-value-bind. How about a function content-type-get which is like a get but in the first position returns the value of the header's content-type.

(defun content-type-get (url) 
  (multiple-value-bind (text http-code http-headers quri x) (dex:get url)
    (values (gethash "content-type" http-headers) 
            text 
            http-code 
            http-header 
            quri 
            x)))

;; this in-addition-content-type-returning get function you can use in return with
;; `multiple-value-bind`. The advantage is that you request the response
;; only once - which ensures speed.

(multiple-value-bind 
  (content-type text http-code http-headers quri x) 
    (content-type-get "http://ip.jsontest.com/")
  (cond ((string= "application/json" content-type) 'do-sth-with-json-text)
        ((string= "text/html; charset=utf-8" content-type) 'do-sth-with-html-text)
        (t 'or-something-else)))

CodePudding user response：

For instance

(defun dex-dispatch (url)
  (multiple-value-bind (content http-result headers uri) (dex:get url)
    ;; cope with errors perhaps here
    (handle-content-type (intern (string-upcase (gethash headers "content-type"))
                                 (find-package "KEYWORD"))
                         content http-result headers uri)))

(defgeneric handle-content-type (content-type content http-result headers uri))

(defmethod handle-content-type :around (content-type content http-result headers uri)
  (declare (ignorable content-type content headers uri))
  (if (= http-result 200)
      (call-next-method)
    ...))

(defmethod handle-content-type ((content-type eql ':application/json)
                               content http-result headers uri)
  ...)

...