formatting

Author: seinecle

docs/generated-html/working-with-text-en.html

@@ -605,7 +605,8 @@ <h2 id="_why_semantic_networks">Why semantic networks?</h2>
 <p>Drawing a semantic network highlights what are the most frequent terms, how they relate to each other, and reveal the different groups or "clusters" of they form.</p>
 </div>
 <div class="paragraph">
-<p>Often, a cluster of terms characterizes a topic. Hence, converting a text into a semantic network helps detecting topics in the text, from micro-topics to the general themes discussed in the documents.</p>
+<p>Often, a cluster of terms characterizes a topic.
+Hence, converting a text into a semantic network helps detecting topics in the text, from micro-topics to the general themes discussed in the documents.</p>
 </div>
 <div class="paragraph">
 <p>Semantic networks are regular networks, where:</p>
@@ -621,7 +622,10 @@ <h2 id="_why_semantic_networks">Why semantic networks?</h2>
 </ul>
 </div>
 <div class="paragraph">
-<p>It means that if you have a textual network, you can visualize it with Gephi just like any other network. Yet, not everything is the same, and this tutorial provides tips and tricks on why textual data can be a bit different than other data.</p>
+<p>It means that if you have a textual network, you can visualize it with Gephi just like any other network.</p>
+</div>
+<div class="paragraph">
+<p>Yet, not everything is the same, and this tutorial provides tips and tricks on why textual data can be a bit different than other data.</p>
 </div>
 </div>
 </div>
@@ -675,7 +679,8 @@ <h4 id="_1_removing_stopwords">1. Removing "stopwords"</h4>
 <div class="sect3">
 <h4 id="_2_considering_n_grams">2. Considering "n-grams"</h4>
 <div class="paragraph">
-<p>So, <code>United States</code> should probably be a meaningful unit, not just <code>United</code> and <code>States</code>. Because <code>United States</code> is composed of 2 terms, it is called a "bi-gram".</p>
+<p>So, <code>United States</code> should probably be a meaningful unit, not just <code>United</code> and <code>States</code>.
+Because <code>United States</code> is composed of 2 terms, it is called a "bi-gram".</p>
 </div>
 <div class="paragraph">
 <p>Trigrams are interesting as well obviously (eg, <code>chocolate ice cream</code>).</p>
@@ -733,7 +738,8 @@ <h4 id="_3_stemming_and_lemmatization">3. Stemming and lemmatization</h4>
 </ul>
 </div>
 <div class="paragraph">
-<p>A tool performing lemmatization is <a href="https://textgrid.de/en/">TextGrid</a>. It has many functions for textual analysis, and lemmatization <a href="https://wiki.de.dariah.eu/display/TextGrid/The+Lemmatizer+Tool">is explained there</a>.</p>
+<p>A tool performing lemmatization is <a href="https://textgrid.de/en/">TextGrid</a>.
+It has many functions for textual analysis, and lemmatization <a href="https://wiki.de.dariah.eu/display/TextGrid/The+Lemmatizer+Tool">is explained there</a>.</p>
 </div>
 </div>
 </div>
@@ -761,7 +767,8 @@ <h2 id="_should_we_represent_all_terms_in_a_semantic_network">Should we represen
 <p>Once this is done, we have transformed the text into plenty of words to represent. Should they all be included in the network?</p>
 </div>
 <div class="paragraph">
-<p>Imagine we have a word appearing just once, in a single footnote of a text long of 2,000 pages. Should this word appear? Probably not.</p>
+<p>Imagine we have a word appearing just once, in a single footnote of a text long of 2,000 pages.
+Should this word appear? Probably not.</p>
 </div>
 <div class="paragraph">
 <p>Which rule to apply to keep or leave out a word?</p>
@@ -782,7 +789,8 @@ <h4 id="_1_start_with_how_many_words_can_fit_in_your_visualization">1. Start wit
 </ul>
 </div>
 <div class="paragraph">
-<p>More words can be crammed in, but in this case the viewer would have to take time zooming in and out, panning to explore the visualization. The viewer transforms into an analyst, instead of a regular reader.</p>
+<p>More words can be crammed in a visualization, but in this case the viewer would have to take time zooming in and out, panning to explore the visualization.
+The viewer transforms into an analyst, instead of a regular reader.</p>
 </div>
 </div>
 <div class="sect3">
@@ -883,7 +891,7 @@ <h2 id="_the_end">the end</h2>
 <div id="footer">
 <div id="footer-text">
 Version 1.0<br>
-Last updated 2017-03-07 22:06:25 CET
+Last updated 2017-03-07 22:17:16 CET
 </div>
 </div>
 </body>

docs/generated-pdf/working-with-text-en.pdf

@@ -25,10 +25,12 @@ This tutorial explains how to draw "semantic networks" like this one:
 image::en/cooccurrences-computer/gephi-result-1-en.png[align="center", title="a semantic network"]
 {nbsp} +
 
+//ST: !
+
 We call "semantic network" a visualization where textual items (words, expressions) are connected to each others, like above.
 
-//ST: !
 We will see in turn:
+//ST: !
 
 - why are semantic networks interesting
 - how to create a semantic network
@@ -37,12 +39,16 @@ We will see in turn:
 
 == Why semantic networks?
 //ST: Why semantic networks?
+//ST: !
 
 A text, or many texts, can be hard to summarize.
 
 Drawing a semantic network highlights what are the most frequent terms, how they relate to each other, and reveal the different groups or "clusters" of they form.
 
-Often, a cluster of terms characterizes a topic. Hence, converting a text into a semantic network helps detecting topics in the text, from micro-topics to the general themes discussed in the documents.
+//ST: !
+
+Often, a cluster of terms characterizes a topic.
+Hence, converting a text into a semantic network helps detecting topics in the text, from micro-topics to the general themes discussed in the documents.
 
 //ST: !
 
@@ -52,15 +58,18 @@ Semantic networks are regular networks, where:
 
 - relations are, usually, signifying a co-occurrences: two words are connected if they co-occur.
 
-It means that if you have a textual network, you can visualize it with Gephi just like any other network. Yet, not everything is the same, and this tutorial provides tips and tricks on why textual data can be a bit different than other data.
-
 //ST: !
+
+It means that if you have a textual network, you can visualize it with Gephi just like any other network.
+
+Yet, not everything is the same, and this tutorial provides tips and tricks on why textual data can be a bit different than other data.
+
 == Choosing what a "term" is in a semantic network
+//ST: Choosing what a "term" is in a semantic network
 //ST: !
 
 The starting point can be: a term is a single word. So in this sentence, we would have 7 terms:
 
-
  My sister lives in the United States (7 words -> 7 terms)
 
 This means that each single term is a meaningful semantic unit.
@@ -90,10 +99,13 @@ You can find a list of these useless terms in many languages, called "stopwords"
 ==== 2. Considering "n-grams"
 //ST: !
 
-So, `United States` should probably be a meaningful unit, not just `United` and `States`. Because `United States` is composed of 2 terms, it is called a "bi-gram".
+So, `United States` should probably be a meaningful unit, not just `United` and `States`.
+Because `United States` is composed of 2 terms, it is called a "bi-gram".
 
 Trigrams are interesting as well obviously (eg, `chocolate ice cream`).
 
+//ST: !
+
 People often stop there, but I find that quadrigrams can be meaningful as well, if less frequent: `United States of America`, `functional magnetic resonance imaging`, `The New York Times`, etc.
 
 Many tools exist to extract n-grams from texts, for example http://homepages.inf.ed.ac.uk/lzhang10/ngram.html[these programs which are under a free license].
@@ -129,11 +141,14 @@ This approach is interesting (implemented for example in the software http://www
 - Stemming consists in chopping the end of the words, so that here, we would have only `live`.
 - Lemmatization is the same, but in a more subtle way: it takes grammar into account. So, "good" and better" would be reduced to "good" because there is the same basic semantic unit behind these two words, even if their lettering differ completely.
 
-A tool performing lemmatization is https://textgrid.de/en/[TextGrid]. It has many functions for textual analysis, and lemmatization https://wiki.de.dariah.eu/display/TextGrid/The+Lemmatizer+Tool[is explained there].
+//ST: !
+
+A tool performing lemmatization is https://textgrid.de/en/[TextGrid].
+It has many functions for textual analysis, and lemmatization https://wiki.de.dariah.eu/display/TextGrid/The+Lemmatizer+Tool[is explained there].
 
 
-//ST: !
 == Should we represent all terms in a semantic network?
+//ST: Should we represent all terms in a semantic network?
 
 //ST: !
 We have seen that some words are more interesting than others in a corpus:
@@ -145,7 +160,8 @@ We have seen that some words are more interesting than others in a corpus:
 //ST: !
 Once this is done, we have transformed the text into plenty of words to represent. Should they all be included in the network?
 
-Imagine we have a word appearing just once, in a single footnote of a text long of 2,000 pages. Should this word appear? Probably not.
+Imagine we have a word appearing just once, in a single footnote of a text long of 2,000 pages.
+Should this word appear? Probably not.
 
 Which rule to apply to keep or leave out a word?
 
@@ -158,7 +174,10 @@ A starting point can be the number of words you would like to see on a visualiza
 - it already fills in all the space of a computer screen.
 - 300 words provides enough information to allow micro-topics of a text to be distinguished
 
-More words can be crammed in, but in this case the viewer would have to take time zooming in and out, panning to explore the visualization. The viewer transforms into an analyst, instead of a regular reader.
+//ST: !
+
+More words can be crammed in a visualization, but in this case the viewer would have to take time zooming in and out, panning to explore the visualization.
+The viewer transforms into an analyst, instead of a regular reader.
 
 //ST: !
 ==== 2. Representing only the most frequent terms
@@ -168,8 +187,8 @@ If ~ 300 words would fit in the visualization of the network, and the text you s
 
 To visualize the semantic network *for a long, single text* the straightforward approach consists in picking the 300 most frequent words (or n-grams, see above).
 
-
 In the case of a collection of texts to visualize (several documents instead of one), two possibilities:
+
 //ST: !
 
 1. Either you also take the most frequent terms across these documents, like before
@@ -213,8 +232,8 @@ tf-idf can be left for specialists of the textual data under consideration, afte
 
 
 == the end
-
 //ST: The end!
+
 Visit https://www.facebook.com/groups/gephi/[the Gephi group on Facebook] to get help,
 
 or visit https://seinecle.github.io/gephi-tutorials/[the website for more tutorials]

docs/generated-slides/subdir/working-with-text-en_temp_common.md

@@ -1,6 +1,6 @@
 = Working with text in Gephi
 Clément Levallois <clementlevallois@gmail.com>
-2017-02-28
+2017-03-07
 
 last modified: {docdate}
 
@@ -95,7 +95,7 @@ So, `United States` should probably be a meaningful unit, not just `United` and
 
 Trigrams are interesting as well obviously (eg, `chocolate ice cream`).
 
-People often stop there, but I find quadrigrams can be meaningful as well, if less frequent: `United States of America`, `functional magnetic resonance imaging`, `The New York Times`, etc.
+People often stop there, but I find that quadrigrams can be meaningful as well, if less frequent: `United States of America`, `functional magnetic resonance imaging`, `The New York Times`, etc.
 
 Many tools exist to extract n-grams from texts, for example http://homepages.inf.ed.ac.uk/lzhang10/ngram.html[these programs which are under a free license].
 
@@ -116,9 +116,9 @@ Take `United States`: it is a noun (`States`) preceded by an adjective (`United`
 
 This approach is interesting (implemented for example in the software http://www.vosviewer.com[Vosviewer]), but it has drawbacks:
 
-- you need to detect adjectives and nouns in your text. This is very language dependent, and slow for large corpora.
+- you need to detect adjectives and nouns in your text. This is language dependent (French put adjectives after nouns, for instance), and the processing is slow for large corpora.
 
-- what about verbs, and noun phrases comprising non adjectives, such as "United States *of* America"? These are not included in the network.
+- what about verbs, and noun phrases comprising non adjectives, such as "United States *of* America"? These are not going to be included in the network.
 
 //ST: !
 [start=3]
@@ -170,7 +170,7 @@ If ~ 300 words would fit in the visualization of the network, and the text you s
 To visualize the semantic network *for a long, single text* the straightforward approach consists in picking the 300 most frequent words (or n-grams, see above).
 
 
-In the case of a colection of texts to visualize (several documents instead of one), two possibilities:
+In the case of a collection of texts to visualize (several documents instead of one), two possibilities:
 //ST: !
 
 1. Either you also take the most frequent terms across these documents, like before
@@ -198,7 +198,7 @@ Applying the tf-idf correction will highlight terms which are frequently used wi
 (to go further, here is a webpage giving a simple example: http://www.tfidf.com/)
 
 //ST: !
-Should you visualize the most frequent words in your corpus, or the words which rank highest according to tf-idf?
+So, should you visualize the most frequent words in your corpus, or the words which rank highest according to tf-idf?
 
 Both are interesting, as they show a different info. I'd suggest that the simple frequency count is easier to interpret.
 
@@ -208,12 +208,10 @@ tf-idf can be left for specialists of the textual data under consideration, afte
 //ST: (to be continued)
 
 
-== More tutorials on importing data to Gephi
-//ST: More tutorials on importing data to Gephi
+== More tutorials on working with semantic networks
+//ST: More tutorials on working with semantic networks
 //ST: !
 
-- https://github.com/gephi/gephi/wiki/Import-CSV-Data[The Gephi wiki on importing csv]
-- https://www.youtube.com/watch?v=3Im7vNRA2ns[Video "How to import a CSV into Gephi" by Jen Golbeck]
 
 == the end
 

docs/generated-slides/subdir/working-with-text-en_temp_html.md

@@ -1,6 +1,6 @@
 = Working with text in Gephi
 Clément Levallois <clementlevallois@gmail.com>
-2017-02-28
+2017-03-07
 
 last modified: {docdate}
 
@@ -95,7 +95,7 @@ So, `United States` should probably be a meaningful unit, not just `United` and
 
 Trigrams are interesting as well obviously (eg, `chocolate ice cream`).
 
-People often stop there, but I find quadrigrams can be meaningful as well, if less frequent: `United States of America`, `functional magnetic resonance imaging`, `The New York Times`, etc.
+People often stop there, but I find that quadrigrams can be meaningful as well, if less frequent: `United States of America`, `functional magnetic resonance imaging`, `The New York Times`, etc.
 
 Many tools exist to extract n-grams from texts, for example http://homepages.inf.ed.ac.uk/lzhang10/ngram.html[these programs which are under a free license].
 
@@ -116,9 +116,9 @@ Take `United States`: it is a noun (`States`) preceded by an adjective (`United`
 
 This approach is interesting (implemented for example in the software http://www.vosviewer.com[Vosviewer]), but it has drawbacks:
 
-- you need to detect adjectives and nouns in your text. This is very language dependent, and slow for large corpora.
+- you need to detect adjectives and nouns in your text. This is language dependent (French put adjectives after nouns, for instance), and the processing is slow for large corpora.
 
-- what about verbs, and noun phrases comprising non adjectives, such as "United States *of* America"? These are not included in the network.
+- what about verbs, and noun phrases comprising non adjectives, such as "United States *of* America"? These are not going to be included in the network.
 
 //ST: !
 [start=3]
@@ -170,7 +170,7 @@ If ~ 300 words would fit in the visualization of the network, and the text you s
 To visualize the semantic network *for a long, single text* the straightforward approach consists in picking the 300 most frequent words (or n-grams, see above).
 
 
-In the case of a colection of texts to visualize (several documents instead of one), two possibilities:
+In the case of a collection of texts to visualize (several documents instead of one), two possibilities:
 //ST: !
 
 1. Either you also take the most frequent terms across these documents, like before
@@ -198,7 +198,7 @@ Applying the tf-idf correction will highlight terms which are frequently used wi
 (to go further, here is a webpage giving a simple example: http://www.tfidf.com/)
 
 //ST: !
-Should you visualize the most frequent words in your corpus, or the words which rank highest according to tf-idf?
+So, should you visualize the most frequent words in your corpus, or the words which rank highest according to tf-idf?
 
 Both are interesting, as they show a different info. I'd suggest that the simple frequency count is easier to interpret.
 
@@ -208,12 +208,10 @@ tf-idf can be left for specialists of the textual data under consideration, afte
 //ST: (to be continued)
 
 
-== More tutorials on importing data to Gephi
-//ST: More tutorials on importing data to Gephi
+== More tutorials on working with semantic networks
+//ST: More tutorials on working with semantic networks
 //ST: !
 
-- https://github.com/gephi/gephi/wiki/Import-CSV-Data[The Gephi wiki on importing csv]
-- https://www.youtube.com/watch?v=3Im7vNRA2ns[Video "How to import a CSV into Gephi" by Jen Golbeck]
 
 == the end