#________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ #_____________________________________________ Definitie discourstypes en vaste variabelen ______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ #________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ #democraticdiscourse = DT1 DT1 = ["democraat", "démocrat", "démocrate", "democraten", "démocrates", "democratie", "démocratie", "démocraties", "démocratique", "démocratiques", "democratisch", "democratische", "démocrats", "referendum", "référendum", "soeverein", "soevereiniteit", "souverain", "souveraineté", "souverains", "souvereine"] #ecologicaldiscourse = DT2 DT2 = ["diversité", "diversiteit", "pluralité", "pluraliteit"] #equalitydiscourse = DT3 DT3 = ["égal", "égale", "égalitair", "égalitaire", "égalité", "égaux", "gelijk", "gelijke", "gelijkheid", "inégal", "inégale", "inégale", "inégales", "inégalité", "inégaux", "inferieur", "inférieur", "inferieure", "inférieure", "inférieures", "inférieurs", "infériorité", "inferioriteit", "ongelijk", "ongelijke", "ongelijkheid", "superieur", "supérieur", "superieure", "supérieure", "supérieures", "supérieurs", "supériorité", "superioriteit"] # mogelijke toevoegingen: avantagé, avantagée, avantagées, avantagés,benadeeld, benadeelde, bevoordeeld, bevoordeelde, défavorable, défavorables, désavantagé, désavantagée, désavantagées, désavantagés, nadelig, nadelige, nadeliger, privilège, privilèges, privilégié, privilégiée, privilégiées, privilégiés #financialdiscourse = DT4 DT4 = ["argent", "besteding", "budget", "budgétaire", "budgétaires", "budgets", "budgettair", "budgettaire", "cher", "chère", "chères", "chers", "dépense", "dépenses", "dure", "duur", "economie", "économie", "économique", "economisch", "economische", "finances", "financiaux", "financieel", "financiel", "financiële", "financielle", "financiels", "financiën", "frais", "geld", "investeren", "investering", "kost", "kosten", "kostprijs", "trésor", "Trésor", "schatkist", "Schatkist"] #historicaldiscourse = DT5 DT5 = ["1830", "geschiedenis", "Guillaume", "histoire", "historie", "historiek", "historique", "historisch", "historische", "Napoleon", "Napoléon", "passé", "Revolutie", "Révolution", "verleden", "Willem"] #legalisticdiscourse = DT6 DT6 = ["bepalingen", "jurisprudence", "jurisprudentie", "legaal", "légal", "legale", "légale", "légales", "légalité", "légaux", "regelgeving", "stipulation", "stipulations", "stipule", "stipulent", "voorschriften"] #liberaldiscourse = DT7 DT7 = ["liberté", "libre", "libres", "vrij", "vrije", "vrijheid"] #manageabilitydiscourse = DT8 DT8 = ["défectueuse", "défectuex", "délai", "délais", "difficile", "difficilement", "difficiles", "difficulté", "difficultés", "doenbaar", "efficace", "efficaces", "efficient", "efficient", "efficiente", "efficiente", "efficientes", "efficientie", "efficients", "facile", "facilement", "faciles", "faisable", "faisables", "haalbaar", "haalbaarheid", "handig", "impossibilité", "impossible", "impossibles", "infaisable", "infaisables", "inréalisible", "inréalisibles", "irréalisable", "irréalisables", "moeilijk", "moeilijke", "moeilijkheden", "moeilijkheid", "mogelijk", "ondoenbaar", "onhandig", "onmogelijk", "onmogelijke", "onmogelijkheid", "onpraktisch", "possible", "possibles", "praktisch", "pratique", "pratiques", "probleem", "problematisch", "problème", "problemen", "problèmes", "réalisable", "réalisables", "réalisible", "réalisibles", "realistisch", "realistische", "vertraging", "vertragingen"] #marxistdiscourse = DT9 DT9 = ["classe", "classes", "klasse", "klassen", "klassenstrijd"] #nationalistdiscourse = DT10 DT10 = ["clan", "clans", "identitaire", "identitaires", "identité", "identiteit", "identités", "natie", "naties", "nation", "nationale", "nationales", "nationalisme", "nationalisme", "nationaliste", "nationalistes", "nationalité", "nationaliteit", "nationalités", "nationaux", "nations", "patrie", "patriot", "patriote", "patriotes", "patriotique", "patriotiques", "patriotisme", "patriotten", "patriottisme", "race", "races", "ras", "rassen", "volk", "volken", "volkeren"] #verwijderd op 5 aug 2017: "étranger", "étrangers", "vreemdeling", "vreemdelingen" #pacifistdiscourse = DT11 DT11 = ["agitatie", "agitation", "agiter", "calme", "calmer", "discorde", "divisé", "divisée", "divisées", "diviser", "divisés", "division", "eenheid", "onrust", "onrustig", "onvrede", "pacifistes", "pacifisme", "pacifiste", "paix", "rust", "tranquil", "tranquille", "tranquilles", "tranquillité", "tranquils", "unité", "verdeeld", "verdeeldheid", "verdelen", "vrede", "vredig"] #rightsdiscourse = DT12 DT12 = ["billijk", "billijkheid", "droit", "droits", "équitable", "équitables", "équité", "gerechtigd", "gerechtigheid", "juste", "justes", "recht", "rechten", "rechtmatig", "rechtmatige", "rechtvaardig", "rechtvaardige", "rechtvaardigheid"] # justice is bewust niet toegevoegd, omdat dit te vaak in andere zin wordt gebruikt #alldiscoursetypes = DT0 DT0 = DT1 + DT2 + DT3 + DT4 + DT5 + DT6 + DT7 + DT8 + DT9 + DT10 + DT11 + DT12 other_standard_keywords = ["abus", "Allemagne", "Allemagne", "Angleterre", "Autriche", "avantagé", "avantagée", "avantagées", "avantagés", "belge", "Belge", "belges", "Belges", "Belgique", "billijke", "citoyen", "citoyenne", "citoyens", "civil", "civile", "civiles", "civils", "conciliation", "constitution", "Constitution", "constitutionelle", "constitutionelles", "crains", "Danemark", "défavorable", "défavorables", "défavorisé", "défavorisée", "défavorisées", "discours", "doit", "également", "équitable", "équitables", "équités", "erreurs", "Espagne", "espère", "étranger", "étrangers", "facultatif", "facultatifs", "facultative", "facultatives", "faculté", "facultés", "flamand", "Flamand", "flamande", "flamands", "Flamands", "Flandre", "France", "fraude", "Grand-Duché", "gratuit", "gratuitement", "gratuits", "grief", "griefs", "grieven", "habitant", "habitants", "habitude", "habitudes", "habituel", "habituels", "honneur", "idée", "idées", "inconvénient", "inconvénients", "indispensable", "indispensables", "individu", "individus", "injuste", "injustes", "injustice", "injustices", "interprètes", "inutile", "inutiles", "inutilité", "Irlande", "Italie", "jurisprudence", "justice", "justices", "langue", "langues", "nécessaire", "nécessaires", "nécessité", "nécessités", "Norwège", "orthographe", "orthographes", "orthographie", "orthographies", "peuple", "peuples", "Pologne", "population", "populations", "possibilité", "possibilités", "principe", "principes", "privilège", "privilèges", "privilégié", "privilégiée", "privilégiées", "privilégiés", "puisse", "réalité", "réclamations", "recueil", "Recueil", "responsabilité", "responsabilités", "responsable", "Russie", "spelling", "Suède", "Suisse", "Suisse", "traducteur", "traducteurs", "transparance", "transparant", "utile", "utiles", "utilité", "vreemdeling", "vreemdelingen", "waal", "waals", "waalse", "walen", "wallon", "Wallon", "Wallonie", "wallonië", "wallonne", "wallonnes", "wallons", "Wallons"] standard_variants = ["interprète", "interprètes", "traducteur", "traducteurs", "traduction", "traductions", "traduira", "traduirai", "traduiraient", "traduirais", "traduirait", "traduiras", "traduire", "traduirez", "traduiriez", "traduirions", "traduirons", "traduiront", "traduis", "traduisaient", "traduisais", "traduisait", "traduisant", "traduise", "traduise", "traduisent", "traduisent", "traduises", "traduisez", "traduisiez", "traduisiez", "traduisîmes", "traduisions", "traduisions", "traduisirent", "traduisis", "traduisisse", "traduisissent", "traduisisses", "traduisissiez", "traduisissions", "traduisit", "traduisît", "traduisîtes", "traduisont", "traduit", "traduite", "traduites", "traduits", "overgezet", "overgezette", "overzetten", "overzetting", "overzettingen", "taalman", "taalmannen", "tolk", "tolken", "traduttore_traditore", "vertaal", "vertaald", "vertaalde", "vertaalden", "vertaalt", "vertalen", "vertaler", "vertalers", "vertaling", "vertalingen", "vertolken", "vertolker", "vertolkers", "vertolking", "vertolkingen", "vertolkt", "vertolkte", "vertolkten"] # Bepaalde van deze termen, vooral van de Nederlandse, staan pro forma in het lijstje, en werden eigenlijk nooit gebruikt, nl.: traduirai, traduiraient, traduirais, traduiras, traduirez, traduirions, traduisaient, traduisais, traduises, traduisiez, traduisîmes, traduisions, traduisirent, traduisis, traduisisse, traduisissent, traduisisses, traduisissiez, traduisissions, traduisît, traduisîtes, traduisont, overgezet, overgezette, overzetten, overzetting, overzettingen, taalman, taalmannen, tolk, tolken, vertaal, vertaald, vertaalde, vertaalden, vertaalt, vertalen, vertaler, vertalers, vertalingen, vertolken, vertolker, vertolkers, vertolking, vertolkingen, vertolkt, vertolkte, vertolkten. #________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ #_____________________________________________ Definitie functie wordfrequencies_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ #________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ def wordfrequencies (startdate="18300101", enddate="19141231", law="none", house="(n)either", job=["parlementslid", "rapporteur", "minister"], names="any", party="any", districts="any", centerword_list=standard_variants, keyword_list=(DT0 + other_standard_keywords), includecenterwordsinkeywords=False, relative=False, printlist=False, order="freq", printrange=False, printplot=False, contextrange="all", minwordlength=2, minwordlengthlist=None, minfrequency=0, ndigits=2): """The wordfrequencies function selects txt-files in the .../Txt's/ directory as input, according to the input factors specified by the user. It outputs range overviews, lists and/or plots, as specified by the user, and saves the plots as pdf's in case they are called. The wordfrequencies function takes the following arguments: | | FILE SELECTION ARGUMENTS: | startdate("yyyymmdd") | What it does: only interventions on a date later than or equal to that startdate are taken into account. | What to input: a date, converted and in quotes | Default value: 1st January 1830 | | enddate("yyyymmdd") | What it does: only interventions on a date prior or equal to that enddate are taken into account. | What to input: a date, converted and in quotes | Default value: 31st December 1914 | | law("yyyy") | What it does: automatically sets the startdate and enddate for analysing discours from the debates prior a | particular law. | What to input: the year this particular law was promulgated (and with which it is associated), in quotes | Default value: "none" (the startdate and enddate as specified are used) | | house("...") | What it does: if specified, only interventions in the Lower or Upper House are taken into account. | What to input: "lower", "upper", "both", "either", "neither", "(n)either" or "n/a" | Default value: (n)either | | job("...") | What it does: only interventions of speakers with the particular function(s) specified, are taken into account. | What to input: "member", "membre", "president", "président", "voorzitter", "rapporteur", "minister", | "member of parliament", "parliamentarian", "mp", "MP", "Kamerlid", "kamerlid", "Senator", | "senator", "parlementslid", "parlementariër", "politician", "politicus", "politicien", | "magistraat", "magistrate", "magistrat", "rechter", "judge", "juge", "jurist", "juriste", | "lawyer", "advocaat", "sollicitor", "barristor", "rechtsgeleerde", "author", "auteur", | "schrijver", "leek", "journalist", "journaliste", "publicist", "writer", "other", or a list of | several of these functions ["…", "…"], or "any". | Default value: "parlementslid", "rapporteur", "minister" (all parliamentary function apart from that of president) | | names("...") | What it does: only interventions of the speakers(s) specified, are taken into account. | What to input: the name of a speaker, without accents and spaces and in quotes (capitals don't matter), or a list | of several speakers ["…", "…"] (The MP Lejeune Vincent is coded LJVincent.) | Default value: any speaker | | party("...") | What it does: only interventions by members of the party/-ies specified, are taken into account. | What to input: "LP", "KP", "BWP", "BOP", "SP", "CD", "MP", "MEETING", Meeting", "Meetingpartij", a list | of several parties ["…", "…"] or "any". | Default value: any party | | disctricts("...") | What it does: only interventions by representatives of the district(s) specified, are taken into account. | What to input: the name of one or more districts (arrondissement) and/or provinces (for Senators after | 1893) without accents and in French or Dutch as according to the language there spoken, or one of the | following codes: "W" for all representatives for French-speaking districts, "V" for all | representatives for Flemish-speaking districts, "D" for all representatives for bilingual | districts with a large German-speaking community, "B" for representatives of the | bilingual city Brussels (and Senators representing the bilingual province of Brabant | after 1893). | Default value: any district | | COMPUTATION ARGUMENTS: | centerword_list("...") | What it does: word(s) considered central in the calculations (in the middle of the contextrange). These | are best selected in accordance with the criteria that was used to compile the corpus of | input files. (For example: the function was designed for research into discourse on | translation, the original input files therefore only included interventions containing that | word or a variant, for which reason the same words were used as central words.) | What to input: a list of any number of words in quotes ["…", "…"] | Default value: a standard list of variants of the words translation and interpreter in French and Dutch, | defined below the definition of the function. | | keyword_list("...") | What it does: words of which the frequency is calculated if ordered to print a list of keywords | (printlist="keywords") | What to input: a list of any number of words in quotes ["…", "…"] | Default value: all keywords associated with one of the 12 discourse types, plus a number of other | special words, previously defined below the definition of the function. | | includecenterwordsinkeywords(...) | What it does: either includes (True) or excludes (False) the central words from the list of keywords, | the frequency of which is calculated if printlist="keywords". | What to input: True or False | Default value: False | | relative(...) | What it does: if True, returns the relative frequency of the keywords (in %), by dividing the absolute | number of their occurrence by the total amount of words taken into account. (If False, | the absolute numbers are returned.) | What to input: True or False | Default value: False | | contextrange(...) | What it does: number that indicates how many words before and after each occurrence of (one of) | the central word(s) should be taken into account. To take into account X words both | before and after each occurrence, the contextrange should be doubled. (For example: | a contextrange of 4 will take into account the 2 words before and after each | occurrence.) Uneven numbers are added by one, in the assumption that the central | words itself was considered part of the total contextrange. (For example: a contextrange | of 5 will also take into account the 2 words before and after each occurrence.) | What to input: an integer between 3 and 199, or "all". | Default value: no limits, the entire interventions in which translation was mentioned are taken into | account. | | minwordlength(...) | What it does: only words with this number of letters or longer are taken into account. | What to input: an integer between 2 and 7 (the number of letters of the shortest centerwords used) | Default value: 2 (all words in the lists) | | OUTPUT ARGUMENTS: | order("...") | What it does: the order by which the list of frequencies is to be printed | What to input: "freq" or "alph" | Default value: sort by frequency (highest frequency first) | | minwordlengthlist(...) | What it does: only words with this number of letters are mentioned, with the frequency of | their occurance, in the frequency list. | What to input: an integer equal to or higher than the set minwordlength (shorter words have a frequency | of 0 anyway, as they are not counted) | Default value: minwordength (same minimum as for the counting of frequencies (in other words: all | words that were considered in the countings are represented in the list of frequencies | (if they appeared at least minfrequency-times)). | | minfrequency(...) | What it does: leaves out keywords from the list that are mentioned fewer times than specified | What to input: a natural number (positive integer) | Default value: 0 (words that are not mentioned are still included in the list) | | ndigits(...) | What it does: relative frequencies are rounded off to this number of decimal places | What to input: a natural number | Default value: 2 decimal places | | printlist(...) | What it does: whether and which kind of list to compose. A list can be composed of the frequencies | of all words taken into account, of the keywords specified or of the words associated | with one the 12 discourse types defined below the definition of the function, | summed for each discourse type. If printlist is set to False, no list is composed. | What to input: "all words", "keywords", "DTs" or False | Default value: False (no list) | | printplot(...) | What it does: whether or not to print a graphical image of the number of frequencies of the 12 | discourse types and (automatically) save it as a PDF in the directory where the | program is run. (Not applicable to frequencies of individual words.) | What to input: True or False | Default value: False | | printrange(...) | What it does: whether or not to (Antconc-wise) print a list of the context around the respective | occurrences of the central word(s), showing the X words before, the central word, | and X words after the central word(s), with X being the contextrange/2. You can | choose to either print no ranges, all ranges or just those ranges | containing at least one of the keywords specified in the keyword_list. | What to input: False, "all" or "keywords" | Default value: False """ from pprint import pprint from json import dumps import matplotlib.pyplot as plt import numpy as np #_______possible problems with input_______# if law == "none": if enddate < startdate: return("Probleem met input: einddatum voor startdatum.") if startdate < "18300101" or startdate >= "19141231": return("Probleem met input: startdatum moet tussen 1 januari 1830 ('18300101') en 30 december 1914 ('19141231') liggen. Let op: aanhalingstekens niet vergeten!") if enddate <= "18300101" or enddate > "19141231": return("Probleem met input: einddatum moet tussen 2 januari 1830 ('18300101') en 31 december 1914 ('19141231') liggen. Let op: aanhalingstekens niet vergeten!") else: if startdate != "18300101" or enddate != "19141231": # Lees: als er naast een specifieke wet ook nog een start- en einddatum zijn opgegeven... return("Probleem met input: indien het onderzoek beperkt wordt tot de debatten naar aanleiding van een specifieke wet, moeten verder geen begin- en einddatum worden opgegeven.") else: if law == "1873": startdate="18730711"; enddate="18730805" elif law == "1889": startdate="18881122"; enddate="18881229" elif law == "1891": startdate="18910703"; enddate="18910807" elif law == "1898": startdate="18961118"; enddate="18980415" elif law == "1908": startdate="19071128"; enddate="19080212" else: return("Probleem met input: gelieve één van volgende afkondigingsjaren op te geven als 'law' (tussen aanhalingstekens): '1873', '1889', '1891', '1898', '1908', of handmatig een specifieke begin- en einddatum in te stellen.") for name in names: if " " in name or "é" in name or "è" in name or "à" in name or "â" in name or "ô" in name: return("Probleem met input. Let op: accenten en spaties in namen moeten worden weggelaten! (Niet hoofdlettergevoelig.)") if minwordlength not in range(2, 8): return("Probleem met input. Minwordlength moet minimaal 2 (lengte van de kortste termen in de tekstbestanden) en maximaal 7 (lengte van het kortste voorkomende varianten van 'traduction'/'interprète'/'vertalen') zijn.") elif minwordlength not in range(2, len(min(keyword_list, key=len))+1): return("Probleem met input. Minwordlength moet minimaal 2 (lengte van de kortste termen in de tekstbestanden) zijn en maximaal de lengte van het korste woord in de keyword_list, om te voorkomen dat het lijkt alsof de korte keywords niet voorkwamen.") districtlist_of_lists = [] if type(districts) == str and districts != "any": districtlist = [districts] if districts != "any": for anydistrict in districts: if anydistrict not in ["V", "W", "B", "D", "any", "Brussel", "Antwerpen", "Mechelen", "Turnhout", "Halle-Vilvoorde", "Leuven", "Brugge", "Kortrijk", "Diksmuide", "Veurne", "Oostende", "Roeselare", "Tielt", "Ieper", "Aalst", "Oudenaarde", "Eeklo", "Gent", "Sint-Niklaas", "Dendermonde", "Hasselt", "Maaseik", "Maastricht", "Tongeren", "Nivelles", "Ath", "Charleroi", "Mons", "Mouscron", "Soignies", "Thuin", "Tournai", "Huy", "Liege", "Verviers", "Waremme", "Arlon", "Bastogne", "Marche-en-Famenne", "Neufchateau", "Virton", "Dinant", "Namur", "Philippeville", "Luxemburg", "Hainaut", "Oost-Vlaanderen", "West-Vlaanderen", "Brabant", "Limburg"]: return("Probleem met input: districts moet gelijk zijn aan 'V' (Vlaanderen), 'W' (Wallonië), 'B' (Brussel), 'D' (Duitstalig), 'any' (kleine letter!), een geldig arrondissement (in de taal aldaar gesproken en tussen aanhalingstekens: 'Antwerpen', 'Mechelen', 'Turnhout', 'Halle-Vilvoorde', 'Leuven', 'Brugge', 'Kortrijk', 'Diksmuide', 'Veurne', 'Oostende', 'Roeselare', 'Tielt', 'Ieper', 'Aalst', 'Oudenaarde', 'Eeklo', 'Gent', 'Sint-Niklaas', 'Dendermonde', 'Hasselt', 'Maaseik', 'Maastricht', 'Tongeren', 'Nivelles', 'Ath', 'Charleroi', 'Mons', 'Mouscron', 'Soignies', 'Thuin', 'Tournai', 'Huy', 'Liege', 'Verviers', 'Waremme', 'Arlon', 'Bastogne', 'Marche-en-Famenne', 'Neufchateau', 'Virton', 'Dinant', 'Namur', 'Philippeville', 'Luxemburg', 'Hainaut', 'Oost-Vlaanderen', 'West-Vlaanderen', 'Brabant', 'Limburg') of een combinatie. (Let op: accenten moeten worden weggelaten!)") elif anydistrict == "B": districtlist_of_lists.append(["Brussel", "Brabant"]) elif anydistrict == "V": districtlist_of_lists.append(["Antwerpen", "Mechelen", "Turnhout", "Halle", "Vilvoorde", "Leuven", "Brugge", "Kortrijk", "Diksmuide", "Veurne", "Oostende", "Roeselare", "Tielt", "Ieper", "Aalst", "Oudenaarde", "Eeklo", "Gent", "Sint-Niklaas", "Dendermonde", "Hasselt", "Maaseik", "Maastricht", "Tongeren", "Oost-Vlaanderen", "West-Vlaanderen", "Limburg"]) elif anydistrict == "W": districtlist_of_lists.append(["Verviers", "Bastogne", "Arlon", "Nivelles", "Ath", "Charleroi", "Mons", "Mouscron", "Soignies", "Thuin", "Tournai", "Huy", "Liege", "Waremme", "Marche-en-Famenne", "Neufchateau", "Virton", "Dinant", "Namur", "Philippeville", "Luxemburg", "Hainaut"]) elif anydistrict == "D": districtlist_of_lists.append(["Verviers", "Bastogne", "Arlon", "Luxemburg"]) else: districtlist_of_lists.append([anydistrict]) districtlist = list(set([district for sublist in districtlist_of_lists for district in sublist])) if type(names) == str: names = [names] if type(job) == tuple: return("Probleem met input. Indien meerdere jobs geselecteerd worden, moet dit gebeuren in een lijst (tussen rechte haken), geen tuple.") joblist_of_lists = [] if type(job) == str: joblist = [job] if job != "any": for anyjob in job: if anyjob in ["member", "membre"]: joblist_of_lists.append(["parlementslid"]) elif anyjob in ["président", "president", "voorzitter"]: joblist_of_lists.append(["voorzitter"]) elif anyjob in ["rapporteur"]: joblist_of_lists.append(["rapporteur"]) elif anyjob in ["ministre", "minister"]: joblist_of_lists.append(["minister"]) elif anyjob in ["member of parliament", "parliamentarian", "mp", "MP", "Kamerlid", "kamerlid", "Senator", "senator", "parlementslid", "parlementariër"]: joblist_of_lists.append(["parlementslid", "voorzitter", "rapporteur", "minister"]) elif anyjob in ["politician", "politicus", "politicien"]: joblist_of_lists.append(["parlementslid", "voorzitter", "rapporteur", "minister", "politicus"]) elif anyjob in ["magistraat", "magistrate", "magistrat", "rechter", "judge", "juge"]: joblist_of_lists.append(["magistrat"]) elif anyjob in ["jurist", "juriste", "lawyer", "advocaat", "sollicitor", "barristor", "rechtsgeleerde"]: joblist_of_lists.append(["jurist"]) elif anyjob in ["author", "auteur", "schrijver", "leek", "journalist", "journaliste", "publicist", "writer"]: joblist_of_lists.append(["author"]) else: return("Probleem met input: variabele 'job' moet gelijk zijn aan 'any', of één van de volgende beroepen: 'member', 'membre', 'président', 'president', 'voorzitter', 'rapporteur', 'minister', 'member of parliament', 'parliamentarian', 'mp', 'MP', 'Kamerlid', 'kamerlid', 'Senator', 'senator', 'parlementslid', 'parlementariër', 'politician', 'politicus', 'politicien', 'magistraat', 'magistrate', 'magistrat', 'rechter', 'judge', 'juge', 'jurist', 'juriste', 'lawyer', 'advocaat', 'sollicitor', 'barristor', 'rechtsgeleerde', 'author', 'auteur', 'schrijver', 'leek', 'journalist', 'journaliste', 'publicist', 'writer' of 'other', of een lijst (geen tuple!) van verschillende van deze beroepen.") joblist = list(set([job for sublist in joblist_of_lists for job in sublist])) if house in ["both", "(n)either", "either", "neither", "n/a"]: house = "(n)either" elif house == "lower": house = "Kamer" elif house == "upper": house = "Senaat" else: return("Probleem met input: variabele 'house' moet gelijk zijn aan 'lower' (Kamer), 'upper' (Senaat), 'both', 'either', 'neither', '(n)either' of 'n/a'.") if all(job in ["parlementslid", "voorzitter", "rapporteur", "minister", "politicus"] for job in joblist) and house == "n/a": return("Probleem met input: indien één van de gespecifieerde jobs een parlementaire functie betreft, moet gespecifieerd worden of het de Kamer (house='lower') of Senaat (house='upper') of beide betreft.") if type(party) == tuple: return("Probleem met input. Indien meerdere partijen geselecteerd worden, moet dit gebeuren in een lijst (tussen rechte haken), geen tuple.") partylist = [] if type(party) == str: partylist = [party] elif party != "any": for anyparty in party: if anyparty in ["POB", "SP"]: partylist.append("BWP") elif anyparty in ["Meeting", "Meetingpartij", "MP"]: partylist.append("MEETING") else: partylist.append(anyparty) for anyparty in partylist: if anyparty not in ["any", "LP", "KP", "BWP", "CD", "MEETING"]: return("Probleem met input. De partij moet 'any', 'LP' (liberale partij), 'KP' (katholieke partij), 'BWP', 'BOP', 'SP' (socialistische partij), 'CD' (christen-democraten), 'MEETING', Meeting' of 'Meetingpartij' zijn, of lijst met verschillende van deze opties. (Merk op: de Meetingpartij en Christen-democraten worden NIET STANDAARD BIJ DE KP geteld. (Aanhalingstekens niet vergeten!)") if printlist == False and printrange == False: return("Let op: aangezien noch een range noch bepaalde frequenties opgevraagd worden, zou er geen enkele output zijn. Wijzig oftewel de waarde van printlist (naar 'all words' of 'keywords') of de waarde printrange (naar 'all' of 'keywords', en erop letten dat de contextrange gelijk is aan een getal onder 200).") if relative not in (True, False): return("Probleem met input: variabele 'relative' moet gelijk zijn aan True of False.") if printlist not in (False, "all words", "keywords", "DTs"): return("Probleem met input: variabele 'printlist' moet gelijk zijn aan False, 'all words', 'keywords' or 'DTs' (discourse types). (Hoofdlettergevoelig!)") if order not in ("freq", "alph"): return("Probleem met input: variabele 'order' moet gelijk zijn aan 'freq' of 'alph'.(Hoofdlettergevoelig!)") if printrange not in (False, "all", "keywords"): return("Probleem met input: variabele 'printrange' moet gelijk zijn aan False, 'all' of 'keywords'.") if contextrange != "all" and contextrange not in range(200): return("Probleem met input: variabele 'contextrange' moet gelijk zijn aan 'all' of een geheel getal onder 200. (Hoofdlettergevoelig!)") if printrange == "all" and contextrange == "all": return("Probleem met input: gelieve een contextrange te specifiëren (bij voorkeur niet te groot), opdat de range geprint zou kunnen worden, of verander de waarde van printrange naar False of 'keywords' (en specifieer in dit laatste geval een beperkt aantal sleutelwoorden in de keyword_list).") if printplot == True and printlist != "DTs": return("Probleem met input: om de plot van de discourstypes te bekomen, moet printlist='DTs'") #_______compilation of word list_______# listoffilenames = ["18301125_Congres_parlementslid_KP_Eeklo_leBegue.txt", "18301125_Congres_parlementslid_KP_Liege_Raikem.txt", "18301125_Congres_parlementslid_KP_Virton_dHuart.txt", "18301125_Congres_parlementslid_LP_Ieper_deLanghe.txt", "18301125_Congres_parlementslid_LP_Maastricht_Destouvelles.txt", "18301125_Congres_parlementslid_LP_Mons_Gendebien.txt", "18301125_Congres_parlementslid_LP_Oudenaarde_Liedts.txt", "18310916_Kamer_minister_KP_Liege_Raikem.txt", "18310916_Kamer_rapporteur_LP_Oostende_Donny.txt", "18311108_Kamer_rapporteur_KP_Hasselt_deTheux.txt", "18370120_Kamer_minister_LP_Liege_Ernst.txt", "18370120_Kamer_parlementslid_KP_Eeklo_Lejeune.txt", "18370120_Kamer_parlementslid_KP_Hasselt_Pollenus.txt", "18370120_Kamer_parlementslid_LP_Mons_Gendebien.txt", "18371201_Kamer_minister_LP_Liege_Ernst.txt", "18371201_Kamer_parlementslid_KP_Eeklo_Lejeune.txt", "18391223_Kamer_minister_KP_Liege_Raikem.txt", "18400530_Kamer_parlementslid_KP_Eeklo_Lejeune.txt", "18440119_Kamer_parlementslid_KP_Tielt_deFoere.txt", "18440125_Kamer_minister_KP_Leuven_dAnethan.txt", "18440125_Kamer_parlementslid_KP_Tielt_deFoere.txt", "18440126_Kamer_minister_KP_Dendermonde_deDecker.txt", "18440126_Kamer_minister_LP_Arlon_Nothomb.txt", "18450117_Kamer_minister_KP_Leuven_dAnethan.txt", "18450117_Kamer_parlementslid_KP_Aalst_Desmet.txt", "18450117_Kamer_parlementslid_KP_Turnhout_Dubus.txt", "18450118_Kamer_minister_KP_Leuven_dAnethan.txt", "18450118_Kamer_parlementslid_KP_Tielt_deFoere.txt", "18450118_Kamer_parlementslid_KP_Turnhout_Dubus.txt", "18450118_Kamer_parlementslid_LP_Brussel_Verhaegen.txt", "18450122_Kamer_minister_KP_Leuven_dAnethan.txt", "18450122_Kamer_parlementslid_KP_Aalst_Desmet.txt", "18450122_Kamer_parlementslid_KP_Roeselare_Rodenbach.txt", "18450122_Kamer_parlementslid_KP_Tielt_deFoere.txt", "18450122_Kamer_parlementslid_KP_Tielt_deRoo.txt", "18450122_Kamer_parlementslid_KP_Tournai_Dumortier.txt", "18570204_Kamer_parlementslid_KP_Sint-Niklaas_deTSerclaes.txt", "18570204_Kamer_parlementslid_LP_Brussel_Orts.txt", "18570205_Kamer_parlementslid_LP_Brugge_Devaux.txt", "18630304_Kamer_parlementslid_KP_Gent_DeBaets.txt", "18630304_Kamer_parlementslid_KP_Sint-Niklaas_VanOverloop.txt", "18630305_Kamer_minister_LP_Arlon_Tesch.txt", "18630305_Kamer_parlementslid_KP_Gent_DeBaets.txt", "18630305_Kamer_parlementslid_KP_Mechelen_Notelteirs.txt", "18630305_Kamer_parlementslid_KP_Sint-Niklaas_VanOverloop.txt", "18660505_Kamer_parlementslid_MEETING_Antwerpen_DeLaet.txt", "18730521_Kamer_parlementslid_MEETING_Antwerpen_DeLaet.txt", "18730711_Kamer_parlementslid_LP_Brussel_Demeur.txt", "18730711_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18730711_Kamer_parlementslid_MEETING_Antwerpen_Delaet.txt", "18730712_Kamer_minister_KP_Diksmuide_deLantsheere.txt", "18730712_Kamer_parlementslid_KP_Aalst_VanWambeke.txt", "18730712_Kamer_parlementslid_KP_Gent_DeBaets.txt", "18730712_Kamer_parlementslid_KP_Gent_Drubbel.txt", "18730712_Kamer_parlementslid_LP_Brussel_Demeur.txt", "18730712_Kamer_parlementslid_LP_Brussel_vanHumbeeck.txt", "18730712_Kamer_parlementslid_LP_Tournai_Bara.txt", "18730712_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18730712_Kamer_parlementslid_MEETING_Antwerpen_Jacobs.txt", "18730712_Kamer_voorzitter_KP_Dinant_Thibaut.txt", "18730715_Kamer_minister_KP_Diksmuide_deLantsheere.txt", "18730715_Kamer_parlementslid_KP_Aalst_VanWambeke.txt", "18730715_Kamer_parlementslid_KP_Gent_DeBaets.txt", "18730715_Kamer_parlementslid_KP_Gent_Delehaye.txt", "18730715_Kamer_parlementslid_KP_Kortrijk_deHaerne.txt", "18730715_Kamer_parlementslid_KP_Leuven_Schollaert.txt", "18730715_Kamer_parlementslid_LP_Brussel_Demeur.txt", "18730715_Kamer_parlementslid_LP_Brussel_Jottrand.txt", "18730715_Kamer_parlementslid_LP_Brussel_vanHumbeeck.txt", "18730715_Kamer_parlementslid_LP_Tournai_Bara.txt", "18730715_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18730715_Kamer_parlementslid_MEETING_Antwerpen_Delaet.txt", "18730715_Kamer_parlementslid_MEETING_Antwerpen_Jacobs.txt", "18730716_Kamer_minister_KP_Diksmuide_deLantsheere.txt", "18730716_Kamer_parlementslid_LP_Brussel_Guillery.txt", "18730716_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18730716_Kamer_parlementslid_MEETING_Antwerpen_Jacobs.txt", "18730717_Kamer_minister_KP_Diksmuide_deLantsheere.txt", "18730717_Kamer_parlementslid_KP_Aalst_VanWambeke.txt", "18730717_Kamer_parlementslid_KP_Leuven_Schollaert.txt", "18730717_Kamer_parlementslid_KP_Namur_Lelievre.txt", "18730717_Kamer_parlementslid_KP_Roeselare_Dumortier.txt", "18730717_Kamer_parlementslid_LP_Brussel_Guillery.txt", "18730717_Kamer_parlementslid_LP_Brussel_vanHumbeeck.txt", "18730717_Kamer_parlementslid_LP_Liege_Muller.txt", "18730717_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18730717_Kamer_parlementslid_MEETING_Antwerpen_Delaet.txt", "18730717_Kamer_voorzitter_KP_Dinant_Thibaut.txt", "18730722_Kamer_minister_KP_Diksmuide_deLantsheere.txt", "18730722_Kamer_parlementslid_KP_Gent_Delehaye.txt", "18730722_Kamer_parlementslid_KP_Gent_Katholiek.txt", "18730722_Kamer_parlementslid_LP_Brussel_Demeur.txt", "18730722_Kamer_parlementslid_LP_Brussel_Guillery.txt", "18730722_Kamer_parlementslid_LP_Charleroi_Pirmez.txt", "18730722_Kamer_parlementslid_LP_Liege_Dupont.txt", "18730722_Kamer_parlementslid_LP_Tournai_Bara.txt", "18730722_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18730722_Kamer_voorzitter_KP_Dinant_Thibaut.txt", "18730723_Kamer_minister_KP_Diksmuide_deLantsheere.txt", "18730723_Kamer_parlementslid_KP_Gent_DeBaets.txt", "18730723_Kamer_parlementslid_KP_Gent_Delehaye.txt", "18730723_Kamer_parlementslid_KP_Gent_Drubbel.txt", "18730723_Kamer_parlementslid_KP_Leuven_Schollaert.txt", "18730723_Kamer_parlementslid_LP_Arlon_Tesch.txt", "18730723_Kamer_parlementslid_LP_Brussel_Demeur.txt", "18730723_Kamer_parlementslid_LP_Brussel_Jottrand.txt", "18730723_Kamer_parlementslid_LP_Charleroi_Pirmez.txt", "18730723_Kamer_parlementslid_LP_Tournai_Bara.txt", "18730723_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18730723_Kamer_parlementslid_MEETING_Antwerpen_Jacobs.txt", "18730723_Kamer_voorzitter_KP_Dinant_Thibaut.txt", "18730724_Kamer_parlementslid_KP_Aalst_VanWambeke.txt", "18730724_Kamer_parlementslid_LP_Brussel_Demeur.txt", "18730724_Kamer_parlementslid_LP_Brussel_vanHumbeeck.txt", "18730724_Kamer_parlementslid_LP_Tournai_Bara.txt", "18730725_Kamer_minister_KP_Diksmuide_deLantsheere.txt", "18730725_Kamer_parlementslid_KP_Leuven_Schollaert.txt", "18730725_Kamer_parlementslid_LP_Brussel_Demeur.txt", "18730725_Kamer_voorzitter_KP_Dinant_Thibaut.txt", "18730805_Senaat_minister_KP_Diksmuide_deLantsheere.txt", "18730805_Senaat_rapporteur_KP_Tielt_dAnethan.txt", "18751119_Kamer_parlementslid_MEETING_Antwerpen_DeLaet.txt", "18761219_Kamer_parlementslid_MEETING_Antwerpen_DeLaet.txt", "18780201_Kamer_parlementslid_KP_Aalst_Woeste.txt", "18780201_Kamer_parlementslid_LP_Mons_Bockstael.txt", "18781218_Kamer_parlementslid_LP_Gent_deVigne.txt", "18790702_Kamer_parlementslid_LP_Gent_deVigne.txt", "18810531_Kamer_parlementslid_MEETING_Antwerpen_Delaet.txt", "18840131_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18861223_Kamer_minister_KP_Hasselt_Thonissen.txt", "18861223_Kamer_parlementslid_LP_Mons_HousseauDeLehaie.txt", "18870120_Kamer_parlementslid_KP_Gent_Begerem.txt", "18880117_Kamer_parlementslid_KP_Gent_Begerem.txt", "18881122_Kamer_minister_KP_Brabant_LeJeune.txt", "18881122_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18881122_Kamer_rapporteur_LP_Liege_Hanssens.txt", "18881123_Kamer_parlementslid_KP_Gent_Eeman.txt", "18881123_Kamer_parlementslid_LP_Liege_Neujean.txt", "18881123_Kamer_parlementslid_LP_Tournai_Bara.txt", "18881123_Kamer_minister_KP_Brabant_LeJeune.txt", "18881123_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18881123_Kamer_rapporteur_LP_Liege_Hanssens.txt", "18881127_Kamer_parlementslid_LP_Ath_deKerchovedeDenterghem.txt", "18881127_Kamer_parlementslid_LP_Huy_Warnant.txt", "18881127_Kamer_minister_KP_Brabant_LeJeune.txt", "18881127_Kamer_parlementslid_MEETING_Antwerpen_Jacobs.txt", "18881128_Kamer_parlementslid_LP_Ath_deKerchovedeDenterghem.txt", "18881128_Kamer_parlementslid_LP_Thuin_AnspachPuissant.txt", "18881128_Kamer_minister_KP_Brabant_LeJeune.txt", "18881128_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18881128_Kamer_rapporteur_LP_Liege_Hanssens.txt", "18881129_Kamer_minister_KP_Brabant_LeJeune.txt", "18881129_Kamer_parlementslid_KP_Aalst_DeSadeleer.txt", "18881129_Kamer_parlementslid_KP_Aalst_Woeste.txt", "18881129_Kamer_parlementslid_KP_Gent_Eeman.txt", "18881129_Kamer_parlementslid_LP_Ath_deKerchovedeDenterghem.txt", "18881129_Kamer_parlementslid_LP_Huy_Warnant.txt", "18881129_Kamer_parlementslid_LP_Liege_Neujean.txt", "18881129_Kamer_parlementslid_LP_Tournai_Bara.txt", "18881129_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18881129_Kamer_voorzitter_KP_Diksmuide_deLantsheere.txt", "18881130_Kamer_parlementslid_LP_Mons_Carlier.txt", "18881130_Kamer_parlementslid_LP_Tournai_Bara.txt", "18881204_Kamer_parlementslid_KP_Aalst_Woeste.txt", "18881204_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18881204_Kamer_voorzitter_KP_Diksmuide_deLantsheere.txt", "18881205_Kamer_parlementslid_KP_Aalst_Woeste.txt", "18881205_Kamer_parlementslid_KP_Leuven_Neeff.txt", "18881205_Kamer_parlementslid_LP_Ath_deKerchovedeDenterghem.txt", "18881205_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18881205_Kamer_voorzitter_KP_Diksmuide_deLantsheere.txt", "18881218_Kamer_minister_KP_Brabant_LeJeune.txt", "18881218_Kamer_parlementslid_KP_Aalst_Woeste.txt", "18881218_Kamer_parlementslid_KP_Ieper_Colaert.txt", "18881218_Kamer_parlementslid_LP_Liege_Neujean.txt", "18881218_Kamer_parlementslid_LP_Tournai_Bara.txt", "18881219_Kamer_parlementslid_KP_Aalst_Woeste.txt", "18881219_Kamer_parlementslid_KP_Brussel_Simons.txt", "18881219_Kamer_parlementslid_KP_Mechelen_Fris.txt", "18881219_Kamer_parlementslid_LP_Liege_Dupont.txt", "18881219_Kamer_parlementslid_LP_Liege_Neujean.txt", "18881219_Kamer_parlementslid_LP_Tournai_Bara.txt", "18881221_Kamer_minister_KP_Brabant_LeJeune.txt", "18881221_Kamer_parlementslid_KP_Brussel_Simons.txt", "18881221_Kamer_parlementslid_LP_Ath_deKerchovedeDenterghem.txt", "18881221_Kamer_parlementslid_LP_Huy_Warnant.txt", "18881221_Kamer_parlementslid_LP_Liege_Neujean.txt", "18881221_Kamer_parlementslid_LP_Tournai_Bara.txt", "18881221_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18881221_Kamer_parlementslid_MEETING_Antwerpen_Jacobs.txt", "18881221_Kamer_voorzitter_KP_Diksmuide_deLantsheere.txt", "18881228_Senaat_parlementslid_KP_Diksmuide_deConinckdeMerckem.txt", "18881228_Senaat_parlementslid_KP_Kortrijk_deBethune.txt", "18881228_Senaat_parlementslid_LP_Waremme_deSelysLongchamps.txt", "18881229_Senaat_minister_KP_Brabant_LeJeune.txt", "18881229_Senaat_parlementslid_KP_Ath_dOultremont.txt", "18881229_Senaat_parlementslid_KP_Brugge_vanOckerhout.txt", "18881229_Senaat_parlementslid_MEETING_Antwerpen_VanPut.txt", "18881229_Senaat_rapporteur_KP_Aalst_VanVreckem.txt", "18910703_Kamer_parlementslid_LP_Liege_Neujean.txt", "18910707_Kamer_parlementslid_KP_Tongeren_Meyers.txt", "18910707_Kamer_parlementslid_LP_Brussel_Graux.txt", "18910807_Senaat_parlementslid_KP_Diksmuide_deConinckdeMerckem.txt", "18910807_Senaat_rapporteur_KP_Kortrijk_Lammens.txt", "18961118_Kamer_parlementslid_BWP_Liege_Anseele.txt", "18961118_Kamer_parlementslid_CD_Aalst_Daens.txt", "18961118_Kamer_parlementslid_KP_Aalst_Woeste.txt", "18961118_Kamer_parlementslid_KP_Gent_VanCleemputte.txt", "18961118_Kamer_parlementslid_KP_Mechelen_VanCauwenbergh.txt", "18961118_Kamer_parlementslid_LP_Virton_Lorand.txt", "18961118_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18961119_Kamer_minister_KP_Gent_Begerem.txt", "18961119_Kamer_parlementslid_KP_Diksmuide_deLantsheere.txt", "18961119_Kamer_parlementslid_KP_Dinant_deMontpellier.txt", "18961119_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18961119_Kamer_voorzitter_KP_Tielt_Beernaert.txt", "18961119_Kamer_rapporteur_KP_Mechelen_VanCauwenbergh.txt", "18970126_Senaat_parlementslid_LP_Philippeville_Tournay.txt", "18970127_Senaat_parlementslid_LP_Liege_Dupont.txt", "18970127_Senaat_parlementslid_MEETING_Antwerpen_Cogels.txt", "18970127_Senaat_rapporteur_KP_Aalst_VanVreckem.txt", "18970128_Senaat_minister_KP_Gent_Begerem.txt", "18970128_Senaat_parlementslid_KP_Antwerpen_LeClef.txt", "18970128_Senaat_parlementslid_KP_Dendermonde_Limpens.txt", "18970128_Senaat_parlementslid_KP_Limburg_Keesen.txt", "18970128_Senaat_parlementslid_KP_Sint-Niklaas_VilainXIIII.txt", "18970128_Senaat_parlementslid_LP_Liege_Magis.txt", "18970129_Senaat_parlementslid_KP_Antwerpen_LeClef.txt", "18970129_Senaat_parlementslid_KP_Dinant_dHuart.txt", "18970129_Senaat_parlementslid_KP_Ieper_SurmontdeVolsberghe.txt", "18970129_Senaat_parlementslid_KP_Luxemburg_Nothomb.txt", "18970129_Senaat_parlementslid_LP_Liege_Dupont.txt", "18970129_Senaat_parlementslid_LP_Liege_MontefioreLevi.txt", "18970202_Senaat_parlementslid_BWP_Hainaut_Picard.txt", "18970202_Senaat_parlementslid_KP_Ieper_SurmontdeVolsberghe.txt", "18970202_Senaat_parlementslid_KP_Oost-Vlaanderen_Leger.txt", "18970203_Senaat_parlementslid_KP_Brussel_VanderBurch.txt", "18970203_Senaat_parlementslid_KP_Kortrijk_deBethune.txt", "18970203_Senaat_parlementslid_KP_Leuven_VandenBossche.txt", "18970203_Senaat_parlementslid_KP_Namur_Poncelet.txt", "18970203_Senaat_parlementslid_LP_Liege_Janson.txt", "18970203_Senaat_parlementslid_LP_Philippeville_Tournay.txt", "18970203_Senaat_parlementslid_LP_Waremme_deSelysLongchamps.txt", "18970203_Senaat_rapporteur_KP_Aalst_VanVreckem.txt", "18970204_Senaat_parlementslid_KP_Kortrijk_Lammens.txt", "18970204_Senaat_parlementslid_KP_Leuven_Roberti.txt", "18970204_Senaat_parlementslid_KP_Neufchateau_Devolder.txt", "18970204_Senaat_parlementslid_KP_West-Vlaanderen_Struye.txt", "18970204_Senaat_parlementslid_LP_Hainaut_Bara.txt", "18970204_Senaat_parlementslid_LP_Liege_Dupont.txt", "18970204_Senaat_parlementslid_LP_Liege_MontefioreLevi.txt", "18970205_Senaat_minister_KP_Gent_Begerem.txt", "18970205_Senaat_parlementslid_BWP_Hainaut_Picard.txt", "18970205_Senaat_parlementslid_KP_Aalst_VanVreckem.txt", "18970205_Senaat_parlementslid_KP_Bastogne_OrbandeXivry.txt", "18970205_Senaat_parlementslid_KP_Brabant_LeJeune.txt", "18970205_Senaat_parlementslid_KP_Brussel_VandenCorput.txt", "18970205_Senaat_parlementslid_KP_Brussel_VanderBurch.txt", "18970205_Senaat_parlementslid_KP_Luxemburg_Nothomb.txt", "18970205_Senaat_parlementslid_KP_Mechelen_dUrsel.txt", "18970205_Senaat_parlementslid_KP_Namur_Poncelet.txt", "18970205_Senaat_parlementslid_LP_Hainaut_Bara.txt", "18970205_Senaat_parlementslid_LP_Liege_Dupont.txt", "18970205_Senaat_parlementslid_LP_Liege_Janson.txt", "18970205_Senaat_parlementslid_LP_Liege_MontefioreLevi.txt", "18970205_Senaat_parlementslid_LP_Verviers_LJVincent.txt", "18980309_Kamer_parlementslid_KP_Brussel_DeVriendt.txt", "18980310_Kamer_parlementslid_BWP_Liege_Jeanne.txt", "18980310_Kamer_parlementslid_KP_Ieper_Colaert.txt", "18980311_Kamer_parlementslid_BWP_Liege_Anseele.txt", "18980311_Kamer_parlementslid_KP_Nivelles_Snoy.txt", "18980311_Kamer_parlementslid_LP_Virton_Lorand.txt", "18980315_Kamer_minister_KP_Gent_Begerem.txt", "18980315_Kamer_parlementslid_KP_Aalst_Woeste.txt", "18980315_Kamer_parlementslid_KP_Arlon_vanLimburgStirum.txt", "18980315_Kamer_parlementslid_KP_Brussel_DeVriendt.txt", "18980315_Kamer_parlementslid_KP_Brussel_VanDerLinden.txt", "18980315_Kamer_parlementslid_KP_Dinant_deMontpellier.txt", "18980316_Kamer_parlementslid_KP_Neufchateau_Heynen.txt", "18980316_Kamer_parlementslid_MEETING_Antwerpen_Coremans.txt", "18980316_Kamer_rapporteur_KP_Mechelen_VanCauwenbergh.txt", "18980317_Kamer_parlementslid_BWP_Liege_Demblon.txt", "18980317_Kamer_parlementslid_CD_Aalst_Daens.txt", "18980317_Kamer_rapporteur_KP_Mechelen_VanCauwenbergh.txt", "18980318_Kamer_parlementslid_BWP_Liege_Smeets.txt", "18980318_Kamer_parlementslid_KP_Nivelles_Snoy.txt", "18980318_Kamer_voorzitter_KP_Aalst_DeSadeleer.txt", "18980318_Kamer_voorzitter_LP_Liege_Magnette.txt", "18980405_Senaat_parlementslid_LP_Liege_Dupont.txt", "18980405_Senaat_parlementslid_LP_Liege_Magis.txt", "18980406_Senaat_parlementslid_KP_Brugge_vanOckerhout.txt", "18980406_Senaat_parlementslid_KP_Kortrijk_Lammens.txt", "18980406_Senaat_parlementslid_KP_Luxemburg_Nothomb.txt", "18980406_Senaat_parlementslid_KP_Sint-Niklaas_VilainXIIII.txt", "18980406_Senaat_parlementslid_LP_Hainaut_Bara.txt", "18980406_Senaat_parlementslid_LP_Liege_Dupont.txt", "18980406_Senaat_parlementslid_LP_Philippeville_Tournay.txt", "18980413_Senaat_parlementslid_BWP_Hainaut_Picard.txt", "18980413_Senaat_parlementslid_KP_Dinant_dHuart.txt", "18980413_Senaat_parlementslid_KP_Luxemburg_Otlet.txt", "18980413_Senaat_parlementslid_KP_Oost-Vlaanderen_Leger.txt", "18980413_Senaat_parlementslid_LP_Hainaut_Bara.txt", "18980413_Senaat_parlementslid_LP_Liege_Dupont.txt", "18980413_Senaat_parlementslid_LP_Waremme_deSelysLongchamps.txt", "18980414_Senaat_parlementslid_BWP_Hainaut_Picard.txt", "18980414_Senaat_parlementslid_KP_Antwerpen_LeClef.txt", "18980414_Senaat_parlementslid_KP_Brussel_Ectors.txt", "18980414_Senaat_parlementslid_KP_Dinant_dHuart.txt", "18980414_Senaat_parlementslid_LP_Liege_Magis.txt", "18980415_Senaat_minister_KP_Gent_Begerem.txt", "18980415_Senaat_parlementslid_KP_Ieper_SurmontdeVolsberghe.txt", "18980415_Senaat_parlementslid_KP_Kortrijk_Lammens.txt", "18980415_Senaat_parlementslid_KP_Limburg_Meyers.txt", "18980415_Senaat_parlementslid_LP_Hainaut_Bara.txt", "18980415_Senaat_parlementslid_LP_Liege_Dupont.txt", "18980415_Senaat_parlementslid_LP_Liege_MontefioreLevi.txt", "18980415_Senaat_parlementslid_LP_Waremme_deSelysLongchamps.txt", "18980413_Senaat_rapporteur_KP_Aalst_VanVreckem.txt", "18980415_Senaat_rapporteur_KP_Aalst_VanVreckem.txt", "19071128_Kamer_minister_CD_Brussel_Renkin.txt", "19071128_Kamer_parlementslid_LP_Antwerpen_Franck.txt", "19071128_Kamer_parlementslid_LP_Brussel_Monville.txt", "19071129_Kamer_minister_CD_Brussel_Renkin.txt", "19071129_Kamer_parlementslid_KP_Brussel_VanDerLinden.txt", "19071129_Kamer_parlementslid_KP_Tournai_Ath_Hoyois.txt", "19071129_Kamer_parlementslid_LP_Brussel_Monville.txt", "19071129_Kamer_rapporteur_KP_Brussel_Wauwermans.txt", "19071204_Kamer_minister_CD_Brussel_Renkin.txt", "19071204_Kamer_parlementslid_BWP_Brussel_Vandervelde.txt", "19071204_Kamer_parlementslid_LP_Brussel_Huismans.txt", "19071204_Kamer_parlementslid_LP_Brussel_Janson.txt", "19071204_Kamer_parlementslid_LP_Brussel_Monville.txt", "19071218_Kamer_minister_CD_Brussel_Renkin.txt", "19071218_Kamer_parlementslid_KP_Verviers_Borboux.txt", "19071218_Kamer_parlementslid_LP_Brussel_Janson.txt", "19071218_Kamer_voorzitter_KP_Leuven_Schollaert.txt", "19080131_Senaat_minister_CD_Brussel_Renkin.txt", "19080131_Senaat_parlementslid_LP_Brussel_Wiener.txt", "19080131_Senaat_parlementslid_LP_Liege_Flechet.txt", "19080211_Senaat_minister_CD_Brussel_Renkin.txt", "19080211_Senaat_minister_KP_Verviers_Simonis.txt", "19080211_Senaat_parlementslid_KP_Dinant_dHuart.txt", "19080211_Senaat_parlementslid_KP_Limburg_Keesen.txt", "19080211_Senaat_parlementslid_KP_Limburg_Meyers.txt", "19080211_Senaat_parlementslid_LP_Antwerpen_VandeWalle.txt", "19080211_Senaat_parlementslid_LP_Brussel_Wiener.txt", "19080212_Senaat_minister_CD_Brussel_Renkin.txt", "19080212_Senaat_parlementslid_BWP_Hainaut_Picard.txt", "19080212_Senaat_parlementslid_KP_Bastogne_OrbandeXivry.txt"] listoffilenameswithspaces = [underscore.replace("_", " ") for underscore in listoffilenames] filenames_cond_startdate = [] for filename in listoffilenameswithspaces: date = ''.join(str(e) for e in [int(s) for s in filename.split() if s.isdigit()]) if date >= startdate: filenames_cond_startdate.append(filename) #print("Aantal tekstbestanden die voldoen aan de voorwaarde startdate: " + str(len(filenames_cond_startdate))) filenames_cond_enddate = [] for filename in listoffilenameswithspaces: date = ''.join(str(e) for e in [int(s) for s in filename.split() if s.isdigit()]) if date <= enddate: filenames_cond_enddate.append(filename) #print("Aantal tekstbestanden die voldoen aan de voorwaarde enddate: " + str(len(filenames_cond_enddate))) filenames_cond_house = [] for filename in listoffilenameswithspaces: if house == "(n)either": filenames_cond_house.append(filename) else: if house in filename: filenames_cond_house.append(filename) #print("Aantal tekstbestanden die voldoen aan de voorwaarde house: " + str(len(filenames_cond_house))) filenames_cond_job = [] if joblist == ["any"]: filenames_cond_job = list(listoffilenameswithspaces) else: for filename in listoffilenameswithspaces: for anyjob in joblist: if anyjob in filename: filenames_cond_job.append(filename) #print("Aantal tekstbestanden die voldoen aan de voorwaarde job: " + str(len(filenames_cond_job))) filenames_cond_names = [] if names == ["any"]: filenames_cond_names = list(listoffilenameswithspaces) else: for filename in listoffilenameswithspaces: for anyname in names: if anyname.lower() in filename.lower(): filenames_cond_names.append(filename) #print("Aantal tekstbestanden die voldoen aan de voorwaarde names: " + str(len(filenames_cond_names))) filenames_cond_party = [] if partylist == ["any"]: filenames_cond_party = list(listoffilenameswithspaces) else: for filename in listoffilenameswithspaces: for anyparty in partylist: if anyparty in filename: filenames_cond_party.append(filename) #print("Aantal tekstbestanden die voldoen aan de voorwaarde party: " + str(len(filenames_cond_party))) filenames_cond_districts = [] if districts == "any" or districts == ['V', 'W', 'B']: filenames_cond_districts = list(listoffilenameswithspaces) else: for filename in listoffilenameswithspaces: for anydistrict in districtlist: if anydistrict in filename: filenames_cond_districts.append(filename) #print("Aantal tekstbestanden die voldoen aan de voorwaarde districts: " + str(len(filenames_cond_districts))) filenames_intersection_frozenset = frozenset(filenames_cond_startdate) & frozenset(filenames_cond_enddate) & frozenset(filenames_cond_job) & frozenset(filenames_cond_house) & frozenset(filenames_cond_names) & frozenset(filenames_cond_party) & frozenset(filenames_cond_districts) filenames_intersection_list = set([filename for filename in filenames_intersection_frozenset]) print("Aantal tekstbestanden die aan alle voorwaarden voldoen: " + str(len((filenames_intersection_list)))) #print(filenames_intersection_list) word_list_of_lists = [] for filename in filenames_intersection_list: file = listoffilenames[listoffilenameswithspaces.index(filename)] sublist = [line.split(',') for line in (open(("Txt's/" + file), "r", encoding="utf-8-sig")).readlines()] word_list_of_lists += sublist word_list_with_separators = [word for sublist in word_list_of_lists for word in sublist] word_list = list(filter(("//").__ne__, word_list_with_separators)) #_______composition of output_______# if printlist == "DTs": list_of_DTs = [DT1, DT2, DT3, DT4, DT5, DT6, DT7, DT8, DT9, DT10, DT11, DT12] list_of_DTs_str = ["DT1", "DT2", "DT3", "DT4", "DT5", "DT6", "DT7", "DT8", "DT9", "DT10", "DT11", "DT12"] list_of_DTs_full = ["democratic discourse", "ecological discourse", "equality discourse", "financial discourse", "historical discourse", "legalistic discourse", "liberal discourse", "manageability discourse", "marxist discourse", "nationalist discourse", "pacifist discourse", "rights discourse"] if printplot == True: DTs_xs = [0, 1, 3, 5, 5, 3, 1, -1, -3, -5, -5, -3, -1]; DTs_ys = [0, 5, 3, 1, -1, -3, -5, -5, -3, -1, 1, 3, 5] DTs_xs_labels = [0, 2, 5, 8, 8, 5, 2, -9, -15, -15, -15, -12, -8]; DTs_ys_labels = [0, 8, 5, 2, -2, -5, -8, -8, -5, -2, 2, 5, 8] if includecenterwordsinkeywords == True: keyword_list += centerword_list if minwordlengthlist == None: minwordlengthlist = int(minwordlength) if contextrange == "all": word_list_x = list(word_list) for word in word_list_x: if len(word) < minwordlength: word_list.remove(word) freq_words = [] for word in word_list: freq_words.append(word_list.count(word)) print("Aantal in rekening gebrachte woorden: ", len(word_list)) if relative == False: dict_of_words = dict(zip(word_list, freq_words)) dict_of_words_minfrequency = {k: v for k, v in dict_of_words.items() if v >= minfrequency and len(k) >= minwordlengthlist} sorted_words_minfrequency = sorted(dict_of_words_minfrequency.items(), key=lambda x:x[1], reverse=True) for keyword in keyword_list: # Toevoeging van de speciale woorden aan de dictionary om foutmeldingen te vermijden. if keyword not in dict_of_words: dict_of_words[keyword] = 0 dict_of_keywords = {k: dict_of_words[k] for k in keyword_list} dict_of_keywords_minfrequency = {k: v for k, v in dict_of_keywords.items() if v >= minfrequency and len(k) >= minwordlengthlist} sorted_keywords_minfrequency = sorted(dict_of_keywords_minfrequency.items(), key=lambda x:x[1], reverse=True) if printlist == "all words": if order == "freq": return("Alle woorden in de volledige teksten, die minstens " + str(minfrequency) + " keer voorkomen, gesorteerd naar frequentie: ", sorted_words_minfrequency) elif order == "alph": return("Alle woorden in de volledige teksten, die minstens " + str(minfrequency) + " keer voorkomen, alfabetisch gesorteerd: ", sorted(dict_of_words_minfrequency.items())) if printlist == "keywords": if order == "freq": return("Sleutelwoorden in de volledige teksten, die minstens " + str(minfrequency) + " keer voorkomen, gesorteerd naar frequentie: ", sorted_keywords_minfrequency) elif order == "alph": return("Sleutelwoorden in de volledige teksten, die minstens " + str(minfrequency) + " keer voorkomen, alfabetisch gesorteerd: ", dumps(dict_of_keywords_minfrequency, sort_keys=True, ensure_ascii=False)) if printlist == "DTs": freq_DT1 = 0; freq_DT2 = 0; freq_DT3 = 0; freq_DT4 = 0; freq_DT5 = 0; freq_DT6 = 0; freq_DT7 = 0; freq_DT8 = 0; freq_DT9 = 0; freq_DT10 = 0; freq_DT11 = 0; freq_DT12 = 0 freq_DT_list = [freq_DT1, freq_DT2, freq_DT3, freq_DT4, freq_DT5, freq_DT6, freq_DT7, freq_DT8, freq_DT9, freq_DT10, freq_DT11, freq_DT12] for word in dict_of_words: for DT in list_of_DTs: if word in DT: freq_DT_list[list_of_DTs.index(DT)] += dict_of_words[word] dict_of_DTs = dict(zip(list_of_DTs_str, freq_DT_list)) if printplot == False: return(sorted(dict_of_DTs.items(), key=lambda x:x[1], reverse=True)) else: print(sorted(dict_of_DTs.items(), key=lambda x:x[1], reverse=True)) freq_DT_list = [0] + freq_DT_list freq_DT_array = np.array(freq_DT_list) DTs_colors = ["w"] for freq in freq_DT_array[1:]: std = np.std(freq_DT_array) av = np.average(freq_DT_array) if freq < (av-std): DTs_colors.append("mistyrose") elif (av-std) <= freq < av: DTs_colors.append("lightcoral") elif av <= freq < (av+std): DTs_colors.append("indianred") elif (av+std) <= freq: DTs_colors.append("firebrick") plt.scatter(x=DTs_xs, y=DTs_ys, s=50*freq_DT_array, c=DTs_colors) plt.xlim(-13, 7); plt.ylim(-8, 8); plt.axis('off'); plt.gca().set_aspect('equal', adjustable='box'); for x, y, label in zip(DTs_xs_labels, DTs_ys_labels, ([party] + list_of_DTs_full)): if x == 0 and y == 0: plt.text(x, y, label, color="firebrick", weight="heavy", fontsize="small") else: plt.text(x, y, label, color="mistyrose", weight="heavy", fontsize="small") plt.savefig(startdate + "-" + enddate + "_house=" + str(house) + "_job=" + str(job) + "_names=" + str(names) + "_districts=" + str(districts) + "_party" + str(party) + "_relative=" + str(relative) + "_range=" + str(contextrange) + "_minwordlength=" + str(minwordlength) + "_contextrange=" + str(contextrange) + ".jpg" , dpi=1200) plt.show() if relative == True: rel_freq_words = [] for number in freq_words: amount_of_words = len(word_list) rel_freq_words.append(round((number/amount_of_words*100), ndigits=ndigits)) dict_of_words_rel = dict(zip(word_list, rel_freq_words)) dict_of_words_minfrequency_rel = {k: v for k, v in dict_of_words_rel.items() if v >= minfrequency and len(k) >= minwordlengthlist} # Merk op dat de meeste frequenties kleiner zijn dan één, en als filter voor relatieve frequenties dus best een minfrequency ongeveer tussen 0,100 en 0,300 genomen wordt.) sorted_words_minfrequency_rel = sorted(dict_of_words_minfrequency_rel.items(), key=lambda x:x[1], reverse=True) for keyword in keyword_list: # Toevoeging van de speciale woorden aan de dictionary om foutmeldingen te vermijden. if keyword not in dict_of_words_rel: dict_of_words_rel[keyword] = 0 dict_of_keywords_rel = {k: dict_of_words_rel[k] for k in keyword_list} dict_of_keywords_minfrequency_rel = {k: v for k, v in dict_of_keywords_rel.items() if v >= minfrequency and len(k) >= minwordlengthlist} sorted_keywords_minfrequency_rel = sorted(dict_of_keywords_minfrequency_rel.items(), key=lambda x:x[1], reverse=True) if printlist == "all words": if order == "freq": return("Alle woorden in de volledige teksten, die minstens " + str(minfrequency) + " keer voorkomen, gesorteerd naar relatieve frequentie: ", sorted_words_minfrequency_rel) elif order == "alph": return("Alle woorden in de volledige teksten, die minstens " + str(minfrequency) + " keer voorkomen, alfabetisch gesorteerd: ", sorted(dict_of_words_minfrequency_rel.items())) if printlist == "keywords": if order == "freq": return("Sleutelwoorden in de volledige teksten, die minstens " + str(minfrequency) + " keer voorkomen, gesorteerd naar relatieve frequentie: ", sorted_keywords_minfrequency_rel) elif order == "alph": return("Sleutelwoorden in de volledige teksten, die minstens " + str(minfrequency) + " keer voorkomen, alfabetisch gesorteerd: ", dumps(dict_of_keywords_minfrequency_rel, sort_keys=True, ensure_ascii=False)) if printlist == "DTs": freq_DT1_rel = 0; freq_DT2_rel = 0; freq_DT3_rel = 0; freq_DT4_rel = 0; freq_DT5_rel = 0; freq_DT6_rel = 0; freq_DT7_rel = 0; freq_DT8_rel = 0; freq_DT9_rel = 0; freq_DT10_rel = 0; freq_DT11_rel = 0; freq_DT12_rel = 0 freq_DT_list = [freq_DT1_rel, freq_DT2_rel, freq_DT3_rel, freq_DT4_rel, freq_DT5_rel, freq_DT6_rel, freq_DT7_rel, freq_DT8_rel, freq_DT9_rel, freq_DT10_rel, freq_DT11_rel, freq_DT12_rel] for word in dict_of_words_rel: for DT in list_of_DTs: if word in DT: freq_DT_list[list_of_DTs.index(DT)] += dict_of_words_rel[word] dict_of_DTs = dict(zip(list_of_DTs_str, freq_DT_list)) if printplot == False: return(sorted(dict_of_DTs.items(), key=lambda x:x[1], reverse=True)) else: print(sorted(dict_of_DTs.items(), key=lambda x:x[1], reverse=True)) freq_DT_list = [0] + freq_DT_list freq_DT_array = np.array(freq_DT_list) DTs_colors = ["w"] for freq in freq_DT_array[1:]: std = np.std(freq_DT_array) av = np.average(freq_DT_array) if freq < (av-std): DTs_colors.append("mistyrose") elif (av-std) <= freq < av: DTs_colors.append("lightcoral") elif av <= freq < (av+std): DTs_colors.append("indianred") elif (av+std) <= freq: DTs_colors.append("firebrick") plt.scatter(x=DTs_xs, y=DTs_ys, s=5000*freq_DT_array, c=DTs_colors) plt.xlim(-13, 13); plt.ylim(-8, 8); plt.axis('off'); plt.gca().set_aspect('equal', adjustable='box'); for x, y, label in zip(DTs_xs_labels, DTs_ys_labels, ([party] + list_of_DTs_full)): if x == 0 and y == 0: plt.text(x, y, label, color="firebrick", weight="heavy", fontsize="small") else: plt.text(x, y, label, color="mistyrose", weight="heavy", fontsize="small") plt.savefig(startdate + "-" + enddate + "_house=" + str(house) + "_job=" + str(job) + "_names=" + str(names) + "_districts=" + str(districts) + "_party" + str(party) + "_relative=" + str(relative) + "_range=" + str(contextrange) + "_minwordlength=" + str(minwordlength) + "_contextrange=" + str(contextrange) + ".jpg", dpi=1200) plt.show() else: word_list_with_separators_x = list(word_list_with_separators) # Zorgt ervoor dat alle woorden korter dan de minwordlength, die meermaals voorkomen worden verwijderd, in plaats van slechts de eerste instantie. for word in word_list_with_separators_x: if len(word) < minwordlength and word != "//": word_list_with_separators.remove(word) word_list = list(word_list_with_separators) words_in_context_with_sublists = [] if contextrange%2 != 0: contextrange -= 1 # Dus als je een oneven contextrange opgeeft, wordt die met 1 verminderd (of m.a.w.: de variant zelf meegeteld, tegenover anders niet). index_word = (contextrange//2)-1 word_list_extended = (contextrange//2) * ["//"] + word_list + (contextrange//2) * ["//"] # Een eindmarkering wordt contextrange/2-keer toegevoegd voor- en achteraan de lijst, om te vermijden dat "list index out of range" bij de berekening van de grenzen. for word in word_list_extended[(contextrange//2):(-(contextrange//2))]: index_word += 1 sublist_with_slashes = [] if word in centerword_list: #print(word) index_left_bound = index_word - (contextrange//2) index_right_bound = index_word + ((contextrange//2)+1) if "//" in word_list_extended[index_left_bound:index_word]: # Indien er een '//' voorkomt in een halve range voor het centerword... #print("Slashes in eerste deel: ", word_list_extended[index_left_bound:index_word]) for i in range(1, ((contextrange//2)+1)): index = index_word - i if word_list_extended[index] == "//": #print(["//"] * len(word_list_extended[index_left_bound:index])) #print(word_list_extended[index:index_word+1]) sublist_with_slashes.append(["//"] * len(word_list_extended[index_left_bound:index])) sublist_with_slashes.append(word_list_extended[index:index_word+1]) #print(len(["//"] * len(word_list_extended[index_left_bound:index])) + len(word_list_extended[index:index_word])) break else: #print("Geen slashes in eerste deel: ", word_list_extended[index_left_bound:index_word+1]) #print(len(word_list_extended[index_left_bound:index_word])) sublist_with_slashes.append(word_list_extended[index_left_bound:index_word+1]) if "//" in word_list_extended[index_word+1:index_right_bound]: # Indien er een '//' voorkomt in een halve range na het centerword... #print("Slashes in tweede deel: ", word_list_extended[index_word+1:index_right_bound]) for i in range(1, ((contextrange//2)+1)): index = index_word + i if word_list_extended[index] == "//": #print(word_list_extended[index_word+1:index]) #print(["//"] * len(word_list_extended[index:index_right_bound])) sublist_with_slashes.append(word_list_extended[index_word+1:index]) sublist_with_slashes.append(["//"] * len(word_list_extended[index:index_right_bound])) #print(len(word_list_extended[index_word+1:index]) + len(["//"] * len(word_list_extended[index:index_right_bound]))) break else: #print("Geen slashes in tweede deel: ", word_list_extended[index_word+1:index_right_bound]) #print(len(word_list_extended[index_word+1:index_right_bound])) sublist_with_slashes.append(word_list_extended[index_word+1:index_right_bound]) #print("") sublist_with_slashes_flat = [word for sublist in sublist_with_slashes for word in sublist] #print(sublist_with_slashes_flat) #print(len(sublist_with_slashes_flat)) words_in_context_with_sublists.append(sublist_with_slashes_flat) #print("") if printrange == "all": ranges_without_slashes = [] for sublist in words_in_context_with_sublists: ranges_without_slashes.append([x for x in sublist if x != "//"]) print(("Termen in context van " + str(contextrange//2) + " woorden voor en na de termen " + "'" + ("', '".join(centerword_list[:-1])) + "'" + " en " + "'" + str(centerword_list[-1]) + "'" + ":")) pprint(ranges_without_slashes) if printrange == "keywords": ranges_without_slashes = [] for sublist in words_in_context_with_sublists: if not set(sublist).isdisjoint(keyword_list): # Als er keywords in de sublists voorkomen (m.a.w. als de keyword_list en sublist elementen gemeen hebben)... ranges_without_slashes.append([x for x in sublist if x != "//"]) print(("Termen in context van " + str(contextrange//2) + " woorden voor en na de termen " + "'" + ("', '".join(centerword_list[:-1])) + "'" + " en " + "'" + str(centerword_list[-1]) + "'" + ":")) pprint(ranges_without_slashes) #print(words_in_context_with_sublists) indexes_centerwords_in_word_list = [] index_list = -1 for word in word_list: index_list += 1 if word in centerword_list: indexes_centerwords_in_word_list.append(index_list) words_in_context_list_of_lists = [] sublist_index = -1 # zal overeenkomen met de index in de word_list van het centerword in het midden van de sublist for sublist in words_in_context_with_sublists: sublist_index += 1 index_previous_centerword = -1 # Stap 1 van het vermijden van overlap: toevoeging van alle woorden in een subset voor en inclusief het middelcenterwoord, tenzij daarvoor nog een ander centerwoord staat, in welk geval slechts de elementen daarachter tot het middencenterwoord worden toegevoegd. index_word = -1 for word in sublist[:(contextrange//2)]: index_word += 1 if word in centerword_list: index_previous_centerword = index_word words_in_context_list_of_lists.append(sublist[(index_previous_centerword + 1):((contextrange//2) + 1)]) # Vanaf hier zal de volgorde van de woorden totaal door elkaar liggen, omdat je eerst van alle sublists het eerste deel neemt, en hierna pas eventueel woorden uit de tweede helft van de respectievelijke sublists. #print("Toevoeging eerste deel: ", sublist[(index_previous_centerword + 1):((contextrange//2) + 1)]) if (words_in_context_with_sublists.index(sublist)+1) == len(words_in_context_with_sublists): # Stap 2a: toevoegen van de tweede helft van de laatste sublist. words_in_context_list_of_lists.append(sublist[((contextrange//2) + 1):]) #print("Laatste sublist, heel tweede helft opgenomen: ", sublist[((contextrange//2) + 1):]) else: more_centerwords_second_half = False # Stap 2b: toevoegen van de tweede helft van de andere sublists of niet, naargelang er geen of wel andere centerwords in staan. for word in sublist[(contextrange//2 + 1):]: if word in centerword_list: more_centerwords_second_half = True if more_centerwords_second_half == False: # Als er in de tweede helft van de sublist geen centerwords staan, moet dit deel nog worden toegevoegd. distance_between_centerwords = indexes_centerwords_in_word_list[sublist_index + 1] - indexes_centerwords_in_word_list[sublist_index] # (let op: moet de afstand in de originele lijst zijn, zonder '//') #print("Afstand checken tot volgende sublist: ", distance_between_centerwords) if "//" in sublist[(contextrange//2 + 1):] or distance_between_centerwords > contextrange or "//" in word_list[indexes_centerwords_in_word_list[sublist_index]:indexes_centerwords_in_word_list[sublist_index + 1]+1]: # Als er voldoende afstand is tot de volgende sublist, of als de zin wordt afgebroken. words_in_context_list_of_lists.append(sublist[((contextrange//2) + 1):]) #print("Einde citaat of voldoende afstand tot volgende sublist: alle woorden tweede helft sublist opgenomen: ", sublist[((contextrange//2) + 1):]) else: words_in_context_list_of_lists.append(sublist[((contextrange//2) + 1):distance_between_centerwords]) # Eindpunt bekomen door (contextrange//2)+(distance_between_centerwords-contextrange//2-1)+1 #print("Volgende sublist nabij. Slechts de woorden die niet overlappen zijn overgenomen: ", sublist[((contextrange//2) + 1):distance_between_centerwords]) #print(words_in_context_list_of_lists) words_in_context = [word for sublist in words_in_context_list_of_lists for word in sublist] words_in_context[:] = [x for x in words_in_context if x != "//"] # Verwijdering van alle "//" uit de woordenlijst, zodat deze de relatieve frequenties niet vertekenen. print("Aantal in rekening gebrachte woorden: ", len(words_in_context)) freq_words_in_context = [] for word in words_in_context: freq_words_in_context.append(words_in_context.count(word)) if relative == False: dict_of_words_in_context = dict(zip(words_in_context, freq_words_in_context)) dict_of_words_minfrequency_in_context = {k: v for k, v in dict_of_words_in_context.items() if v >= minfrequency and len(k) >= minwordlengthlist} sorted_words_minfrequency_in_context = sorted(dict_of_words_minfrequency_in_context.items(), key=lambda x:x[1], reverse=True) for keyword in keyword_list: # Toevoeging van de speciale woorden aan de dictionary om foutmeldingen te vermijden. if keyword not in dict_of_words_in_context: dict_of_words_in_context[keyword] = 0 dict_of_keywords_in_context = {k: dict_of_words_in_context[k] for k in keyword_list} dict_of_keywords_minfrequency_in_context = {k: v for k, v in dict_of_keywords_in_context.items() if v >= minfrequency and len(k) >= minwordlengthlist} sorted_keywords_minfrequency_in_context = sorted(dict_of_keywords_minfrequency_in_context.items(), key=lambda x:x[1], reverse=True) if printlist == "all words": if order == "freq": return("Alle woorden in de context, die minstens " + str(minfrequency) + " keer voorkomen, gesorteerd naar frequentie: ", sorted_words_minfrequency_in_context) elif order == "alph": return("Alle woorden in de context, die minstens " + str(minfrequency) + " keer voorkomen, alfabetisch gesorteerd: ", sorted(dict_of_words_minfrequency_in_context.items())) if printlist == "keywords": if order == "freq": return("Sleutelwoorden in de context, die minstens " + str(minfrequency) + " keer voorkomen, gesorteerd naar frequentie: ", sorted_keywords_minfrequency_in_context) elif order == "alph": return("Sleutelwoorden in de context, die minstens " + str(minfrequency) + " keer voorkomen, alfabetisch gesorteerd: ", dumps(dict_of_keywords_minfrequency_in_context, sort_keys=True, ensure_ascii=False)) if printlist == "DTs": freq_DT1_in_context = 0; freq_DT2_in_context = 0; freq_DT3_in_context = 0; freq_DT4_in_context = 0; freq_DT5_in_context = 0; freq_DT6_in_context = 0; freq_DT7_in_context = 0; freq_DT8_in_context = 0; freq_DT9_in_context = 0; freq_DT10_in_context = 0; freq_DT11_in_context = 0; freq_DT12_in_context = 0 freq_DT_list = [freq_DT1_in_context, freq_DT2_in_context, freq_DT3_in_context, freq_DT4_in_context, freq_DT5_in_context, freq_DT6_in_context, freq_DT7_in_context, freq_DT8_in_context, freq_DT9_in_context, freq_DT10_in_context, freq_DT11_in_context, freq_DT12_in_context] for word in dict_of_words_in_context: for DT in list_of_DTs: if word in DT: freq_DT_list[list_of_DTs.index(DT)] += dict_of_words_in_context[word] dict_of_DTs = dict(zip(list_of_DTs_str, freq_DT_list)) if printplot == False: return(sorted(dict_of_DTs.items(), key=lambda x:x[1], reverse=True)) else: print(sorted(dict_of_DTs.items(), key=lambda x:x[1], reverse=True)) freq_DT_list = [0] + freq_DT_list freq_DT_array = np.array(freq_DT_list) DTs_colors = ["w"] for freq in freq_DT_array[1:]: std = np.std(freq_DT_array) av = np.average(freq_DT_array) if freq < (av-std): DTs_colors.append("mistyrose") elif (av-std) <= freq < av: DTs_colors.append("lightcoral") elif av <= freq < (av+std): DTs_colors.append("indianred") elif (av+std) <= freq: DTs_colors.append("firebrick") plt.scatter(x=DTs_xs, y=DTs_ys, s=50*freq_DT_array, c=DTs_colors) plt.xlim(-13, 13); plt.ylim(-8, 8); plt.axis('off'); plt.gca().set_aspect('equal', adjustable='box') for x, y, label in zip(DTs_xs_labels, DTs_ys_labels, ([party] + list_of_DTs_full)): if x == 0 and y == 0: plt.text(x, y, label, color="firebrick", weight="heavy", fontsize="small") else: plt.text(x, y, label, color="mistyrose", weight="heavy", fontsize="small") plt.savefig(startdate + "-" + enddate + "_house=" + str(house) + "_job=" + str(job) + "_names=" + str(names) + "_districts=" + str(districts) + "_party" + str(party) + "_relative=" + str(relative) + "_range=" + str(contextrange) + "_minwordlength=" + str(minwordlength) + "_contextrange=" + str(contextrange) + ".jpg", dpi=1200) plt.show() if relative == True: rel_freq_words_in_context = [] for number in freq_words_in_context: amount_of_words_in_context = len(words_in_context) rel_freq_words_in_context.append(round((number/amount_of_words_in_context*100), ndigits=ndigits)) dict_of_words_in_context_rel = dict(zip(words_in_context, rel_freq_words_in_context)) dict_of_words_minfrequency_in_context_rel = {k: v for k, v in dict_of_words_in_context_rel.items() if v >= minfrequency and len(k) >= minwordlengthlist} sorted_word_minfrequency_in_context_rel = sorted(dict_of_words_minfrequency_in_context_rel.items(), key=lambda x:x[1], reverse=True) for keyword in keyword_list: # Toevoeging van de speciale woorden aan de dictionary om foutmeldingen te vermijden. if keyword not in dict_of_words_in_context_rel: dict_of_words_in_context_rel[keyword] = 0 dict_of_keywords_in_context_rel = {k: dict_of_words_in_context_rel[k] for k in keyword_list} dict_of_keywords_minfrequency_in_context_rel = {k: v for k, v in dict_of_keywords_in_context_rel.items() if v >= minfrequency and len(k) >= minwordlengthlist} sorted_keyword_minfrequency_in_context_rel = sorted(dict_of_keywords_minfrequency_in_context_rel.items(), key=lambda x:x[1], reverse=True) if printlist == "all words": if order == "freq": return("Alle woorden in de context, die minstens " + str(minfrequency) + " keer voorkomen, gesorteerd naar relatieve frequentie: ", sorted_word_minfrequency_in_context_rel) elif order == "alph": return("Alle woorden in de context, die minstens " + str(minfrequency) + " keer voorkomen, alfabetisch gesorteerd: ", sorted(dict_of_words_minfrequency_in_context_rel.items())) if printlist == "keywords": if order == "freq": return("Sleutelwoorden in de context, die minstens " + str(minfrequency) + " keer voorkomen, gesorteerd naar relatieve frequentie: ", sorted_keyword_minfrequency_in_context_rel) elif order == "alph": return("Sleutelwoorden in de context, die minstens " + str(minfrequency) + " keer voorkomen, alfabetisch gesorteerd: ", dumps(dict_of_keywords_minfrequency_in_context_rel, sort_keys=True, ensure_ascii=False)) if printlist == "DTs": freq_DT1_in_context_rel = 0; freq_DT2_in_context_rel = 0; freq_DT3_in_context_rel = 0; freq_DT4_in_context_rel = 0; freq_DT5_in_context_rel = 0; freq_DT6_in_context_rel = 0; freq_DT7_in_context_rel = 0; freq_DT8_in_context_rel = 0; freq_DT9_in_context_rel = 0; freq_DT10_in_context_rel = 0; freq_DT11_in_context_rel = 0; freq_DT12_in_context_rel = 0 freq_DT_list = [freq_DT1_in_context_rel, freq_DT2_in_context_rel, freq_DT3_in_context_rel, freq_DT4_in_context_rel, freq_DT5_in_context_rel, freq_DT6_in_context_rel, freq_DT7_in_context_rel, freq_DT8_in_context_rel, freq_DT9_in_context_rel, freq_DT10_in_context_rel, freq_DT11_in_context_rel, freq_DT12_in_context_rel] for word in dict_of_words_in_context_rel: for DT in list_of_DTs: if word in DT: freq_DT_list[list_of_DTs.index(DT)] += dict_of_words_in_context_rel[word] dict_of_DTs = dict(zip(list_of_DTs_str, freq_DT_list)) if printplot == False: return(sorted(dict_of_DTs.items(), key=lambda x:x[1], reverse=True)) else: print(sorted(dict_of_DTs.items(), key=lambda x:x[1], reverse=True)) freq_DT_list = [0] + freq_DT_list freq_DT_array = np.array(freq_DT_list) DTs_colors = ["w"] for freq in freq_DT_array[1:]: std = np.std(freq_DT_array) av = np.average(freq_DT_array) if freq < (av-std): DTs_colors.append("mistyrose") elif (av-std) <= freq < av: DTs_colors.append("lightcoral") elif av <= freq < (av+std): DTs_colors.append("indianred") elif (av+std) <= freq: DTs_colors.append("firebrick") plt.scatter(x=DTs_xs, y=DTs_ys, s=5000*freq_DT_array, c=DTs_colors) plt.xlim(-13, 13); plt.ylim(-8, 8); plt.axis('off'); plt.gca().set_aspect('equal', adjustable='box'); for x, y, label in zip(DTs_xs_labels, DTs_ys_labels, ([party] + list_of_DTs_full)): if x == 0 and y == 0: plt.text(x, y, label, color="firebrick", weight="heavy", fontsize="small") else: plt.text(x, y, label, color="mistyrose", weight="heavy", fontsize="small") plt.savefig(startdate + "-" + enddate + "_house=" + str(house) + "_job=" + str(job) + "_names=" + str(names) + "_districts=" + str(districts) + "_party" + str(party) + "_relative=" + str(relative) + "_range=" + str(contextrange) + "_minwordlength=" + str(minwordlength) + "_contextrange=" + str(contextrange) + ".jpg", dpi=1200) plt.show() #________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ #_____________________________________________ Gewenste output __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ #________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ #outputfilename = "20180313_platonique_perregio" #with open("C:/Users/u0106296/Box Sync/Word frequency tool/Output/" + str(outputfilename) + ".txt", "w") as text_file: # for party in ["BWP", "LP", "KP", "MEETING", "CD"]: # for regio in ["B", "W", "V", "D"]: # #for contextrange in ("all", 100): # for absrel in [True, False]: # text_file.write("partij: " + str('alle partijen') + ", regio: " + str(regio) + ", contextrange: " + str("all") + ", absoluut of relatief: " + str(absrel) + ":" + "\n") # text_file.write(str(wordfrequencies(law="1898", ndigits=3, printlist="keywords", party=["BWP", "LP", "KP", "MEETING", "CD"], keyword_list=["platonique", "platoniques"], contextrange="all", minwordlength=2, minfrequency=0, relative=absrel, districts=regio, order="alph"))) # text_file.write(2*"\n")