What happens when you train a neural network to write investment treaties? Below are the results (paper).
Note: This application uses the entirety of a country’s treaty practice as benchmark. Predicted output may thus not necessarily correspond to a country’s most recent treaty design.
We employed a 2-layer LSTM
torch-rnn with 512 nodes per layer, sequence length of 200 characters and a dropout factor of 0.5 to train it on 80% of the data on 1626 English-language delineated bilateral investment treaties (10% were used for validation and test sets) for each issue area corpus separately. Corpus texts were constructed by concatenating split article texts back to one large text file, preserving the article numbers and names in headers, which precede each article text within each treaty. We trained each model for 10 epochs (case with priors) or 50 epochs (case without priors). Validation set loss was typically lower than train set loss due to a high dropout factor specified, signalling little overfitting. Then we specified the starting sequence of “#Article” (signifies a new article delimiter in train data) and generated 150 strings of 100,000 characters from the trained model with a temperature of 0.5 (a factor between 0 and 1 by which the predicted character probabilities are divided to supply more innovative results) for each issue area.
“With thousands of treaties, many ongoing negotiations and multiple dispute-settlement mechanisms, today’s IIA regime has come close to a point where it is too big and complex to handle for governments and investors alike.”
Using text-as-data analysis, we reduce this complexity and allow policy-makers, arbitrators, and scholars to:
Scholars and arbitrators have recognized that common principles underlie investment treaties. Amongst others, BITs typically provide for:
At the same time, scholars and arbitrators have noted that treaties diverge in their individual wording. The lack of adequate empirical tools, however, has long made it difficult to quantify just how different or how similar these treaties are.
Mapping BITs now remedies this shortcoming allowing users to discover uniformity and diversity among BITs for themselves.
The large heat map compares 1628 English-language BITs concluded between 1959 and 2014. Every treaty occurs once on the horizontal axis and once on the vertical axis. We've developed a continuous metric to gauge similarity between treaties (see Methodology section for the details). This metric ranges from 0 (dark red, full similarity) to 1 (bright yellow, no similarity):
Each treaty is compared with itself along the diagonal line. The two sides of the heat map above and below that line are symmetric. Black quadrangles are the borders of individual country treaty networks. Alternatively, they delimit the clusters of similar treaties.
For each bilateral treaty, we identified the wealthier treaty party based on its GDP per capita at the time of signature. Where a treaty involves an OECD member, or, alternatively, a BRIC country, that country is always named first as wealthier party by default.
Therefore, you can sort the treaties on the axes heat map in two ways: by wealthier party or by less wealthy counterpart. Within parties, the treaties are always sorted by date of signature. In addition to that, we've identified clusters of similar treaties. By choosing the third option you sort the treaties by clusters.
We show the world map and color countries depending on their engagement in BIT network. We've obtained the data on the universe of BITs signed from UNCTAD. The more treaties a country signed, the darker is the shade of blue. However, we have English-language treaty texts for 51% of the treaties ever signed. The potential undersampling is displayed with red color palette.
Finally, we rank the countries based on the coherence of their treaty networks. The lower the mean distance between treaties struck by a country, the darker is the shade of green on the map for this country.
Click on a country to view a heat map of its BIT network. Each field of the heat map represents two BITs being compared. You can reorder the heat map to match your research interest:
We also provide links to UNCTAD country profiles to give users background information on countries' economies, international trade and FDI.
Our study reveals the following insights:
Alschner, W. and D. Skougarevskiy (2015). Consistency and Legal Innovation in the BIT Universe. Stanford Public Law and Legal Theory Working Paper Series No. 2595288. http://ssrn.com/abstract=2595288
Alschner, W. and D. Skougarevskiy (2015). The new gold standard? Empirically Situating the TPP in the Investment Treaty universe. Centre for Trade and Economic Integration Working Paper No. 2015-08. http://graduateinstitute.ch/files/live/sites/iheid/files/sites/ctei/shared/CTEI/working_papers/CTEI%202015-8%20Alschner_Skougarevskiy_TPP.pdf
This research has benefitted from the funding and support of the following grants and projects:
This website breaks new ground by mapping 1628 English language investment treaties – 51% of the BIT universe. For our TPP special we augment the data with Investment Chapter texts from 51 Free Trade Agreement, and 7 Multilateral Investment Treaties.
Second, we manually edit these texts, remove side-letters and schedules of reservations and correct typos, optical character recognition errors, and other mistakes in underlying data sources. We also unify the treaty spelling, converting all British English words to their American English counterparts (e.g. “favour” to “favor”).
Third, we transform text into data by computing treaty differences. Every treaty is dissected into its constituent 5-character substrings (5-grams) preserving word order:
“shall not be permitted” → “shall”, “hall_”, “all_n”, “ll_no”, “l_not”, “_not_”, “not_b”, “ot_be”, “t_be_“, “_be_ p”, “be_ pe”, “e_ per”, “_ perm”, “permi”, “ermit”, “rmitt”, “mitte”, “itted”
Suppose there are two treaties: one containing a phrase “shall not be permitted” and second containing a phrase “shall be permitted”. Their 5-character substrings will be:
|“shall not be permitted”||“shall be permitted”|
We then compare these treaties by the extent to which they share 5-character substrings. The above two treaties have 21 unique 5-grams, of which 11 feature in both treaties, or 52%. Substracting this figure from 100% yields 48%, a measure of dissimilarity between two treaties. Formally, this is known as the Jaccard distance, which ranges between 0 and 1:
The resulting 1628×1628 distance matrix stores all bilateral similarities.
Fifth, we apply Affinity propagation clustering to the dissimilarity matrix in order to uncover structure in the BIT universe. We set the preference q (the a priori suitability of a point to serve as an exemplar) to the lowest quantile of the similarities (minimum similarity). That produces 94 clusters.
Sixth, we come up with a way to interactively display the dissimilarities, on a heat map. It compares BITs based on their similarities (dark red) and differences (bright yellow).
Seventh, for each treaty we locate its 20 closest neighbours in terms of Jaccard distance. We also uncover the cleavages between treaties in a pair by comparing word usage. To this end, we construct a document-term matrix from the treaties (stop words are removed, British spelling differences are mitigated with VarCon). With the aid of the document-term matrix we compute proportions of word use for each word in each treaty. Applying the two-proportion z-test, we find the words that are 10% significantly differently used in the selected treaty pair and display them in an interactive chart.
Eighth, we examine the subsets of treaties by country and compute internal coherence of signed treaties for each country. This enables us to rank countries based on their treaty network coherence. We exclude countries that struck less than 4 treaties from this ranking to ensure that the results are not driven by treaty network size.
Wolfgang is a post-doctoral researcher at the World Trade Institute and at the Graduate Institute of International and Development Studies.
Before obtaining his Ph.D. at the Graduate Institute and JSM at Stanford Law School, Wolfgang worked for the Institute's International Law Department as Professor Joost Pauwelyn's research and teaching assistant. He was, amongst others, responsible for the courses ‘International Trade Law’, ‘International Investment Law’ and the ‘International Trade and Investment Law Clinic’. Wolfgang also acted repeatedly as academic advisor to the Graduate Institute WTO Summer Programme.
Dmitriy is a Ph.D. candidate at The Graduate Institute of International and Development Studies and a researcher at the Institute for the Rule of Law at the European University at St. Petersburg.
He works at the intersection of Law and Economics, applying economic analysis to various fields of law and seeking to answer legal questions with hard data. Dmitriy is active in sentencing research and economics of crime studies.
When using for non-commercial purposes, please quote as Alschner, Wolfgang and Dmitriy Skougarevskiy. (2015) Mapping BITs (http://mappinginvestmenttreaties.com).
Data on the BIT universe is from the UNCTAD’s Investment Policy Hub. English-language BIT texts used to compute Jaccard distances come from Kluwer Arbitration, Investment Claims, and UNCTAD’s Investment Policy Hub. Texts displayed in word diff tables are from Investment Policy Hub only.