Skip to content

Commit 23efaa2

Browse files
cmungallcmungall
committed
switch to docgen
1 parent 184b89e commit 23efaa2

15 files changed

Lines changed: 702 additions & 689 deletions

‎Makefile‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -186,11 +186,13 @@ download/reactome-biopax.zip:
186186

187187
MODULES = rdf owl obo omo relation_graph semsql
188188

189+
GENDOC_ARGS = --no-mergeimports -d docs --template-directory docgen-templates
190+
189191
# TODO: markdown gen should make modular output
190192
markdown-%: $(YAML_DIR)/%.yaml
191-
$(RUN) gen-markdown --no-mergeimports -d docs $< && mv docs/index.md docs/$*_index.md
193+
$(RUN) gen-doc $(GENDOC_ARGS) $< && mv docs/index.md docs/$*_index.md
192194
markdown: $(patsubst %, markdown-%, $(MODULES))
193-
$(RUN) gen-markdown --no-mergeimports -d docs $(YAML_DIR)/semsql.yaml
195+
$(RUN) gen-doc $(GENDOC_ARGS) $(YAML_DIR)/semsql.yaml
194196

195197
gen-project: $(YAML_DIR)/semsql.yaml
196198
$(RUN) gen-project $< -d project

‎README.md‎

Lines changed: 36 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,44 @@
44
![](https://github.com/incatools/semantic-sql/workflows/Build/badge.svg)
55

66

7-
This project provides a standard collection of SQL tables/views for ontologies, such that you can make queries like:
7+
This project provides a standard collection of SQL tables/views for ontologies, such that you can make queries like this,
8+
to find all terms starting with `Abnormality` in [HPO](https://obofoundry.org/ontology/hp).
89

910
```sql
10-
SELECT * FROM rdfs_label_statement WHERE value LIKE 'Abnormality of %';
11+
$ sqlite db/hp.db
12+
sqlite> SELECT * FROM rdfs_label_statement WHERE value LIKE 'Abnormality of %';
1113
```
1214

13-
Ready-made ontologies can also be downloaded for any ontology in [OBO](http://obofoundry.org)
15+
|stanza|subject|predicate|object|value|datatype|language|
16+
|---|---|---|---|---|---|---|
17+
|HP:0000002|HP:0000002|rdfs:label||Abnormality of body height|xsd:string||
18+
|HP:0000014|HP:0000014|rdfs:label||Abnormality of the bladder|xsd:string||
19+
|HP:0000022|HP:0000022|rdfs:label||Abnormality of male internal genitalia|xsd:string||
20+
|HP:0000032|HP:0000032|rdfs:label||Abnormality of male external genitalia|xsd:string||
21+
22+
23+
Ready-made ontologies can also be downloaded for any ontology in [OBO](http://obofoundry.org), using URLs such as https://s3.amazonaws.com/bbop-sqlite/hp.db
24+
25+
[relation-graph](https://github.com/balhoff/relation-graph/) is used to pre-generate tables of [entailed edges](https://incatools.github.io/semantic-sql/EntailedEdge/). For example,
26+
all is-a and part-of ancestors of [finger](http://purl.obolibrary.org/obo/UBERON_0002389) in Uberon:
27+
28+
sqlite> SELECT * FROM entailed_edge WHERE subject='UBERON:0002389' and predicate IN ('rdfs:subClassOf', 'BFO:0000050');
29+
1430

1531
## Installation
1632

33+
SemSQL comes with a helper Python library. Use of this is optional. To install:
34+
1735
```bash
1836
pip install semsql
1937
```
2038

2139
## Download ready-made SQLite databases
2240

41+
Pre-generated SQLite database are created weekly for all OBO ontologies and a selection of others.
42+
43+
To download:
44+
2345
```bash
2446
semsql download obi -o obi.db
2547
```
@@ -40,14 +62,19 @@ In either case:
4062
- The input MUST be in RDF/XML serialization and have the suffix `.owl`:
4163
- use robot to convert if format is different
4264

65+
We are planning to simplify this process in future.
66+
4367
### 1. Build a SQLite database directly
4468

69+
This requires some basic technical knowledge about how to install things on your machine
70+
and how to put things in your PATH. It does not require Docker.
71+
4572
Requirements:
4673

4774
- [rdftab.rs](https://github.com/ontodev/rdftab.rs)
4875
- [relation-graph](https://github.com/balhoff/relation-graph)
4976

50-
After installing these and putting both in your path:
77+
After installing these and putting both `relation-graph` and `rdftab.rs` in your path:
5178

5279
```bash
5380
semsql make foo.db
@@ -70,15 +97,16 @@ docker run -v $PWD:/work -w /work -ti linkml/semantic-sql semsql make foo.db
7097

7198
## Schema
7299

73-
See [LinkML Schema Docs](https://incatools.github.io/semantic-sql/)
100+
See [Schema Documentation](https://incatools.github.io/semantic-sql/)
74101

75-
The source schema is in LinkML - this is then compiled down to SQL Tables and Views
102+
The [source schema](https://github.com/INCATools/semantic-sql/tree/main/src/semsql/linkml) is in [LinkML](https://linkml.io) - this is then compiled down to SQL Tables and Views
76103

77104
## ORM Layer
78105

79106
A SemSQL relational database can be accessed in exactly the same way as any other SQLdb
80107

81-
For convenience, we provide a Python ORM layer using SQL Alchemy. This allows code like the following, which joins [RdfsSubclassOfStatement](https://incatools.github.io/semantic-sql/RdfsSubclassOfStatement) and [existential restrictions](https://incatools.github.io/semantic-sql/OwlSomeValuesFrom):
108+
For convenience, we provide a Python Object-Relational Mapping (ORM) layer using SQL Alchemy.
109+
This allows for code uchlike the following, which joins [RdfsSubclassOfStatement](https://incatools.github.io/semantic-sql/RdfsSubclassOfStatement) and [existential restrictions](https://incatools.github.io/semantic-sql/OwlSomeValuesFrom):
82110

83111
```python
84112
engine = create_engine(f"sqlite:////path/to/go.db")
@@ -113,4 +141,5 @@ You can also pass in an OWL file and have the sqlite be made on the fly
113141
runoak -i sqlite:envo.owl search t~biome
114142
```
115143

144+
Even if using OAK, it can be useful to access SQL tables directly to do complex multi-join queries in a performant way.
116145

‎docgen-templates/class.md.jinja2‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
# Class: {{ gen.name(element) }}
2+
3+
{%- if header -%}
4+
{{header}}
5+
{%- endif -%}
6+
7+
8+
{% if element.description %}
9+
_{{ element.description }}_
10+
{% endif %}
11+
12+
{% if element.abstract %}
13+
* __NOTE__: this is an abstract class and should not be instantiated directly
14+
{% endif %}
15+
{% if element.mixin %}
16+
* __NOTE__: this is a mixin class intended to be used in combination with other classes, and not used directly
17+
{% endif %}
18+
19+
URI: {{ gen.uri_link(element) }}
20+
21+
22+
23+
{% if schemaview.class_parents(element.name) %}
24+
```{{ gen.mermaid_directive() }}
25+
classDiagram
26+
{% for s in schemaview.class_parents(element.name)|sort(attribute='name') -%}
27+
{{ gen.name(schemaview.get_element(s)) }} <|-- {{ gen.name(element) }}
28+
{% endfor %}
29+
{% for s in schemaview.class_induced_slots(element.name)|sort(attribute='name') -%}
30+
{{ gen.name(element) }} : {{gen.name(s)}}
31+
{% endfor %}
32+
33+
```
34+
{% elif schemaview.class_children(element.name) %}
35+
```{{ gen.mermaid_directive() }}
36+
classDiagram
37+
{% for s in schemaview.class_children(element.name)|sort(attribute='name') -%}
38+
{{ gen.name(element) }} <|-- {{ gen.name(schemaview.get_element(s)) }}
39+
{% endfor %}
40+
{% for s in schemaview.class_induced_slots(element.name)|sort(attribute='name') -%}
41+
{{ gen.name(element) }} : {{gen.name(s)}}
42+
{% endfor %}
43+
```
44+
{% else %}
45+
```{{ gen.mermaid_directive() }}
46+
classDiagram
47+
class {{ gen.name(element) }}
48+
{% for s in schemaview.class_induced_slots(element.name)|sort(attribute='name') -%}
49+
{{ gen.name(element) }} : {{gen.name(s)}}
50+
{% endfor %}
51+
```
52+
{% endif %}
53+
54+
55+
{% if schemaview.class_parents(element.name) or schemaview.class_children(element.name, mixins=False) %}
56+
57+
## Usage
58+
59+
```sql
60+
SELECT * FROM {{element.name}};
61+
```
62+
63+
## Inheritance
64+
{{ gen.inheritance_tree(element, mixins=True) }}
65+
{% else %}
66+
<!-- no inheritance hierarchy -->
67+
{% endif %}
68+
69+
## Slots
70+
71+
| Name | Cardinality and Range | Description |
72+
| --- | --- | --- |
73+
{% for s in schemaview.class_induced_slots(element.name) -%}
74+
| {{gen.link(s)}} | {{ gen.cardinality(s) }} <br/> {{gen.link(s.range)}} | {{s.description|enshorten}} |
75+
{% endfor %}
76+
77+
## Usages
78+
79+
{% if schemaview.usage_index().get(element.name) %}
80+
| used by | used in | type | used |
81+
| --- | --- | --- | --- |
82+
{% for usage in schemaview.usage_index().get(element.name) -%}
83+
| {{gen.link(usage.used_by)}} | {{gen.link(usage.slot)}} | {{usage.metaslot}} | {{usage.used }} |
84+
{% endfor %}
85+
{% endif %}
86+
87+
{% include "common_metadata.md.jinja2" %}
88+
89+
{% if element.classification_rules %}
90+
## Classification Rules
91+
92+
This class has classification rules. This allows this class (table) to be derived
93+
from another class (table) via a query.
94+
95+
{% for r in element.classification_rules %}
96+
* Parent: {{gen.link(r.is_a)}}
97+
* Conditions:
98+
{% for sc in r.slot_conditions.values() %}
99+
* {{gen.link(sc.name)}} = {{sc.equals_string}}
100+
{% endfor %}
101+
{% endfor %}
102+
103+
{% endif %}
104+
105+
{% if element.rules or element.classification_rules %}
106+
107+
## Rules
108+
109+
{% endif %}
110+
111+
{% for comment in element.comments %}
112+
{% if comment.startswith("sqlview>>") %}
113+
## SQL View
114+
115+
This class has a SQL view definition:
116+
117+
```
118+
{{comment}}
119+
```
120+
121+
{% endif %}
122+
{% endfor %}
123+
124+
{% if schemaview.get_mappings(element.name).items() -%}
125+
## Mappings
126+
127+
| Mapping Type | Mapped Value |
128+
| --- | --- |
129+
{% for m, mt in schemaview.get_mappings(element.name).items() -%}
130+
{% if mt|length > 0 -%}
131+
| {{ m }} | {{ mt }} |
132+
{% endif -%}
133+
{% endfor %}
134+
135+
{% endif -%}
136+
137+
138+
## LinkML Specification
139+
140+
<!-- TODO: investigate https://stackoverflow.com/questions/37606292/how-to-create-tabbed-code-blocks-in-mkdocs-or-sphinx -->
141+
142+
### Direct
143+
144+
<details>
145+
```yaml
146+
{{gen.yaml(element)}}
147+
```
148+
</details>
149+
150+
### Induced
151+
152+
<details>
153+
```yaml
154+
{{gen.yaml(element, inferred=True)}}
155+
```
156+
</details>
157+
158+
{%- if footer -%}
159+
{{footer}}
160+
{%- endif -%}

‎docgen-templates/common_metadata.md.jinja2‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
{% if element.comments -%}
2+
## Comments
3+
4+
{% for x in element.comments -%}
5+
* {{x}}
6+
{% endfor %}
7+
{% endif -%}
8+
9+
{% if element.todos -%}
10+
## TODOs
11+
12+
{% for x in element.todos -%}
13+
* {{x}}
14+
{% endfor %}
15+
{% endif -%}
16+
17+
## Identifier and Mapping Information
18+
19+
{% if element.id_prefixes %}
20+
### Valid ID Prefixes
21+
22+
Instances of this class *should* have identifiers with one of the following prefixes:
23+
{% for p in element.id_prefixes %}
24+
* {{p}}
25+
{% endfor %}
26+
27+
{% endif %}
28+
29+
30+
{% if element.annotations %}
31+
### Annotations
32+
33+
| property | value |
34+
| --- | --- |
35+
{% for a in element.annotations -%}
36+
| {{a}} | {{ element.annotations[a].value }} |
37+
{% endfor %}
38+
{% endif %}
39+
40+
{% if element.from_schema or element.imported_from %}
41+
### Schema Source
42+
43+
{% if element.from_schema %}
44+
* from schema: {{ element.from_schema }}
45+
{% endif %}
46+
{% if element.imported_from %}
47+
* imported from: {{ element.imported_from }}
48+
{% endif %}
49+
{% endif %}

‎docgen-templates/enum.md.jinja2‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# {{ gen.name(element) }}
2+
3+
{{ element.description }}
4+
5+
URI: {{ gen.uri(element) }}
6+
7+
{% if element.permissible_values -%}
8+
## Permissible Values
9+
10+
| Value | Meaning | Description | Info |
11+
| --- | --- | --- | --- |
12+
{% for pv in element.permissible_values.values() -%}
13+
| {{pv.text}} | {{pv.meaning}} | {{pv.description|enshorten}} | |
14+
{% endfor %}
15+
{% else %}
16+
_This is a dynamic enum_
17+
{% endif %}
18+
19+
{% include "common_metadata.md.jinja2" %}
20+
21+
## Schema
22+
23+
<details>
24+
```yaml
25+
{{gen.yaml(element)}}
26+
```
27+
</details>
28+

‎docgen-templates/index.md.jinja2‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# {% if schema.title %}{{ schema.title }}{% else %}{{ schema.name }}{% endif %}
2+
3+
{{ schema.description }}
4+
5+
URI: {{ schema.id }}
6+
Name: {{ schema.name }}
7+
8+
## Classes
9+
10+
| Class | Description |
11+
| --- | --- |
12+
{% for c in gen.all_class_objects()|sort(attribute=sort_by) -%}
13+
| {{gen.link(c)}} | {{c.description|enshorten}} |
14+
{% endfor %}
15+
16+
## Slots
17+
18+
| Slot | Description |
19+
| --- | --- |
20+
{% for s in gen.all_slot_objects()|sort(attribute=sort_by) -%}
21+
| {{gen.link(s)}} | {{s.description|enshorten}} |
22+
{% endfor %}
23+
24+
## Enumerations
25+
26+
| Enumeration | Description |
27+
| --- | --- |
28+
{% for e in gen.all_enum_objects()|sort(attribute=sort_by) -%}
29+
| {{gen.link(e)}} | {{e.description|enshorten}} |
30+
{% endfor %}
31+
32+
## Types
33+
34+
| Type | Description |
35+
| --- | --- |
36+
{% for t in gen.all_type_objects()|sort(attribute=sort_by) -%}
37+
| {{gen.link(t)}} | {{t.description|enshorten}} |
38+
{% endfor %}
39+
40+
## Subsets
41+
42+
| Subset | Description |
43+
| --- | --- |
44+
{% for ss in schemaview.all_subsets().values()|sort(attribute='name') -%}
45+
| {{gen.link(ss)}} | {{ss.description|enshorten}} |
46+
{% endfor %}

0 commit comments

Comments
 (0)