Tools for converting data into schema.
(Still very much pre-alpha!)
Implemented in multiple languages.
| Index | Coverage | Supported versions | Downloads | |
|---|---|---|---|---|
| Python |  | - | ||
| JavaScript (node.js) |  | |||
| Rust (coming soon!) | - | - | - | - |
| Nim (coming soon!) | - | - | - | - |
You can install this from the pypi index. It's available as the derek-py package.
Simple example with pip (poetry is recommended):
pip install derek-pyComplete set of supported installation methods:
| Package manager | pypi | git |
|---|---|---|
| pip | pip install derek-py | pip install git+https://github.com/benjaminwoods/derek@main |
| poetry | poetry add derek-py | poetry add git+https://github.com/benjaminwoods/derek#main |
You can install this from the npm index. It's available as the derek-ts package.
Simple example with yarn:
yarn add derek-tsComplete set of supported installation methods:
| Package manager | npm | git |
|---|---|---|
| npm | npm i derek-ts | npm i git+https://github.com/benjaminwoods/derek#main |
| yarn | yarn add derek-ts | yarn add git+https://github.com/benjaminwoods/derek#main |
Here's a quick guide showing what you can do with derek. These examples are for a Python implementation.
Load some data into a tree of nodes:
# Import the main class from derek import Derek # Suppose that you have some JSON-compatible data obj = [ { 'some': [1.0, 3, "4.5"], 'data': [3.4, 4.5] }, { 'some': [2, "4.0", 1.5], 'data': [1.4] } ] # Feed this data into Derek.tree root_node = Derek.tree(obj, name='MyDataStructure')You can use .example() to see a simple example item of data:
>>> root_node.example() [{'some': [1.0], 'data': [3.4]}]You can produce an OAS2/OAS3 JSON schema from this data, too:
j = root_node.parse(format='oas3') import json print(json.dumps(j, indent=2)){ "MyDataStructure": { "type": "array", "items": { "type": "object", "additionalProperties": { "oneOf": [ { "type": "array", "items": { "oneOf": [ { "type": "string" }, { "type": "integer" }, { "type": "number" } ] } }, { "type": "array", "items": { "type": "number" } } ] } }, "example": [ { "some": [1.0], "data": [3.4] } ] } }Install and use the yaml package to convert this structure to an OAS3-compliant data schema.
import yaml print(yaml.dump(j))MyDataStructure: example: - data: - 3.4 some: - 1.0 items: additionalProperties: oneOf: - items: type: number type: array - items: oneOf: - type: number - type: integer - type: string type: array type: object type: arrayQuickly extract schemas from APIs, by feeding the returned JSON into Derek.
from derek import Derek from pycoingecko import CoinGeckoAPI cg = CoinGeckoAPI() # Get all coins from CoinGecko root_node = Derek.tree(cg.get_coins_list(), name='GetCoins')Parse to get your schema:
j = root_node.parse(format='oas3') import json print(json.dumps(j, indent=2)){ "GetCoins": { "type": "array", "items": { "type": "object", "additionalProperties": { "type": "string" } }, "example": [ { "id": "01coin", "symbol": "zoc", "name": "01coin" } ] } }No required dependencies. Always.
Use libraries like pywhat and yaml to quickly extend Derek:
import json, yaml from derek import Derek, Parser from pywhat import Identifier class PywhatDerek(Derek): @property def parser(self): return PywhatParser() def get_oas3_yaml(self): return yaml.dump( self.parse(format="oas3") ) class PywhatParser(Parser): @classmethod def oas2(cls, node): # Call the superclass parser for the current node: # _sup = cls.__mro__[PywhatParser.__mro__.index(int):] # j = _sup.oas2(cls, node) # All calls to the oas2 method in the superclass therefore re-route # back to this class method, automatically handling all recursive calls # here. j = super(PywhatParser, cls).oas2(node) # The rest of this function simply patches in results from a call # to the pywhat API. identifier = Identifier() if all(map(lambda t: not isinstance(node.value, t), [list, dict])): result = identifier.identify(str(node.value)) if result['Regexes'] is not None: matches = [entry for entry in result['Regexes']['text']] # Select the match as the longest string map_func = lambda d: (d['Matched'], d['Regex Pattern']['Name']) max_func = lambda tup: len(tup[0]) _, match = max( map(map_func, matches), key=max_func ) j = { **j, 'description': match } return jAllowing for functionality like:
root_node = PywhatDerek.tree( {'data': ['17VZNX1SN5NtKa8UQFxwQbFeFc3iqRYhem']}, name='Addresses' ) root_node.get_oas3_yaml()returning:
Addresses: additionalProperties: items: description: "Bitcoin (\u20BF) Wallet Address" type: string type: array example: data: - 17VZNX1SN5NtKa8UQFxwQbFeFc3iqRYhem type: objectDerek is designed for ease of use. If you're trying to use Derek functionality in a workflow and it feels like it should be easier to get your desired result, please make an issue.