This week we will be talking about text analysis in class, also known as text mining. This is simply the process of deriving high quality information from text. But how does this benefit us in our field of history, and how do you even start this process? Ted Underwood gives a simple account of what text analysis can do and how to get started in his article Where To Start With Text Mining.
To begin to gather data, you will first need to have a good selection of quality readable information. As Underwood mentions in his article, a great deal of this information can be found on JSTOR. JSTOR is short for Journal Storage, founded in 1995, this is a massive storage of academic journals, primary sources, and now books are beginning to find their way into the collection. Around 80% of this is usable data that does not need to be translated, however, there are some readings that you cannot pull text directly from due to its limited transferability due to poor scan quality of the actual text. There are several programs being developed that can correct this problem, which brings the amount of transferability up to 98% in some cases.
The result of the text mining and the quantitative methods that it requires can benefit us in several ways, the result can be a number of different ways to categorize information, contrast the vocabulary of different texts, trace the history of different words or phrases over time, cluster features associated in different documents, entity extraction, and visualization of data. This information can help historians understand what a word or phrase has meant in the past and how that information has changed. In short, a word used today could've meant something totally different 80 years ago. This information helps us research-wise to insure that we are fully understanding the past when we read it in historical accounts.
After Google launched its web search API in 2002, Roy Rosenweig took it upon himself to develop a similar search tool that could be used on documents in a "document classification," or a large number of texts and syllabi. This ability to search a large number of documents pertaining to a subject would allow us as researchers a quick and easy tool to search for a phrase used in these documents. Clemson's library and many of its databases utilize this technology and has made it quite simple to search amongst the hundreds of thousands of documents stored within the databases.
Utilizing the information extracted from these documents that are researched, historians can then develop visualization. This is particularly important for the younger generations, as people today have a significantly less attention span when it comes to reading data as generations past. With the development of technology, there is no need to have people read through large amounts of information when a visual aid can provide the same information in a single picture. This also helps historians convey history in a simple and concise manner that will make a larger impact on the reader. Finally, I feel like history is being brought to life and making it much more understandable to people outside of the history profession. This in turn is making much more interested in history, instead of dreading the class because all they see is dates, people, and places.
**UPDATE ON DIGITAL HISTORY PROJECT**
The groundwork has been laid to get the Fort Hill project in motion. Information is being developed so that this particular site can be viewed digitally and can provide research assistance to individuals that cannot make the trek to Clemson's campus. This in turn can provide Clemson with the ability to advertise its history on the internet, bringing attention to some of the things Clemson has to offer.
Wednesday, September 25, 2013
Wednesday, September 18, 2013
Mapping the world, or our perspective?
Looking at a map can tell an individual many things, it can depict changes in terrain, show roads, and identify cities, but what else does it tell us? After reading How Maps Lie by Leah Fleckenstein, I have a whole new perspective on maps and their meanings. To say all new perspective, I have to be honest, the only perspective I've ever had on maps was utilizing them to commute through states and cities, and to manage the terrain in Iraq and Afghanistan. After joining the military, I should've recognized the difference that maps can make from one person or another. For example, if you were to tell an Iraqi that has lived in Al Anbar Province his entire life to take Route Michigan to Camp Korean Village, he would look at you as if you were mad. The simple act of renaming towns and streets can change a persons perspective on a place, whether one admits it or not.
The entire process of developing a map can be both arduous and time consuming, however, the due diligence must be done in order for accuracy to be achieved. Over the years maps have become more and more sophisticated and the ability to manufacture these maps have become easier and easier thanks to satellite capabilities. But how can these maps of the past help us determine the pop culture of the time it was written? The ability to manipulate maps has been perfected over the years, especially during times of war. Utilizing a map and using certain colors the map maker can imply falsities and can trick people into believing one thing versus the truth. The Nazis utilized this very practice to gain favor within Germany.
I'm more interested in utilizing maps in order to show the progression of a society. Using things such as maps and graphs have given historians the ability to visually transmit history. We've only been in this class for a few weeks, and I've already found new ways of showing history to individuals that may not be as interested in it. As we continue class, my partner and I have narrowed down our project and it will include an elaborate view of Fort Hill Plantation, and will (hopefully) allow people outside the state to visually see the house and its progression over the years. Many pictures will be taken in the weeks to come, and I need to meet with a web designer to figure out the process of making the project a success. I will post pictures of the progression as I see fit, after all, I don't want to ruin the surprise. Stay tuned.
-Jason
The entire process of developing a map can be both arduous and time consuming, however, the due diligence must be done in order for accuracy to be achieved. Over the years maps have become more and more sophisticated and the ability to manufacture these maps have become easier and easier thanks to satellite capabilities. But how can these maps of the past help us determine the pop culture of the time it was written? The ability to manipulate maps has been perfected over the years, especially during times of war. Utilizing a map and using certain colors the map maker can imply falsities and can trick people into believing one thing versus the truth. The Nazis utilized this very practice to gain favor within Germany.
I'm more interested in utilizing maps in order to show the progression of a society. Using things such as maps and graphs have given historians the ability to visually transmit history. We've only been in this class for a few weeks, and I've already found new ways of showing history to individuals that may not be as interested in it. As we continue class, my partner and I have narrowed down our project and it will include an elaborate view of Fort Hill Plantation, and will (hopefully) allow people outside the state to visually see the house and its progression over the years. Many pictures will be taken in the weeks to come, and I need to meet with a web designer to figure out the process of making the project a success. I will post pictures of the progression as I see fit, after all, I don't want to ruin the surprise. Stay tuned.
-Jason
Wednesday, September 11, 2013
The Space Between: The Importance of a Historian and Geographer Collaboration
Today is 9/11 and I can't help but have most of my thoughts instinctively focused towards the events that happened on this day twelve years ago. My Facebook feed is littered with digital documentation of that day and its after effects that it had on the lives of many Americans. While I can distinctively recall my exact location when I first saw what was happening on the news, I can't help but turn my perspective to how important this digital collection will be to humanity when I'm in the ground and merely a memory to those in my family. These videos and audio files have documented in great length the history of that day, the first time in history that an event like this has been documented in such great detail.
The role of the historian and the role of the geographer are rarely seen as one in the same. While historians slave over journal entries and newspaper articles, geographers are deeply imbedding themselves in vast amounts of information from days throughout history. Both of these professions have the same goal, to tell the story of the world, but the two go at it from two entirely different points of view. Historians seem to latch themselves to the personal lives of people that have lived, while geographers steer themselves in the direction of the state of the world the people lived in. The question that begs an answer is, why aren't there more collaborative teams that are composed of historians and geographers?
While the two professions often find a common ground in large facilities such as the Smithsonian Institute and large colleges, the benefits of this common ground are not fully being integrated into the educational syllabus of the schools and colleges. In high school I took geography one year, which was mainly comprised of the shape of the states and where rivers are, the following year I took a history class that was comprised of memorizing states, dates, and names. Is it any wonder that our education of the past is sorely lacking? When I first started college, I quickly realized just how much I'd been cheated. As years pass and historians of newer generations make their way into professor positions around the country, the focus of many history departments has departed from the "memorization" game and has headed into the "cause and effect" game. An individual might be able to learn dates, places, and names, but to be educated on the importance of this information is more important. In the age of smart phones and tablets, a specific date an event occurred can be searched and found in a matter of seconds.
Technology has afforded these two professions a second chance at happiness. Historic maps can now be imbedded in a program that can stream seamlessly with Google Maps. Sure, the technology is new and there are a few bugs to work out, but imagine the importance of this system once the vast amounts of information is imputed. Imagine the ability to scroll over a building in NYC and having the ability to click on it, which would bring up the building's history in regards to pictures, dates, building permits, and other historic information. Imagine having the ability to run a program that shows westward expansion to your children visually. Imagine having the ability to see the geographic changes of the world combined with personal stories and information of its residents. While the technology is a long way off as of now, but it is never too early to start. When you see something from a singular point of view, its like looking at life in 2D. However, looking at something from multiple points of view utilizing the full extent of knowledge, suddenly you are looking at life in 3D. Geographers have the ability to do there job without historians, however, I don't feel that historians can do there job without geographers. The importance of the vast data is too relevant to be ignored.
-Jason Barrett
The role of the historian and the role of the geographer are rarely seen as one in the same. While historians slave over journal entries and newspaper articles, geographers are deeply imbedding themselves in vast amounts of information from days throughout history. Both of these professions have the same goal, to tell the story of the world, but the two go at it from two entirely different points of view. Historians seem to latch themselves to the personal lives of people that have lived, while geographers steer themselves in the direction of the state of the world the people lived in. The question that begs an answer is, why aren't there more collaborative teams that are composed of historians and geographers?
While the two professions often find a common ground in large facilities such as the Smithsonian Institute and large colleges, the benefits of this common ground are not fully being integrated into the educational syllabus of the schools and colleges. In high school I took geography one year, which was mainly comprised of the shape of the states and where rivers are, the following year I took a history class that was comprised of memorizing states, dates, and names. Is it any wonder that our education of the past is sorely lacking? When I first started college, I quickly realized just how much I'd been cheated. As years pass and historians of newer generations make their way into professor positions around the country, the focus of many history departments has departed from the "memorization" game and has headed into the "cause and effect" game. An individual might be able to learn dates, places, and names, but to be educated on the importance of this information is more important. In the age of smart phones and tablets, a specific date an event occurred can be searched and found in a matter of seconds.
Technology has afforded these two professions a second chance at happiness. Historic maps can now be imbedded in a program that can stream seamlessly with Google Maps. Sure, the technology is new and there are a few bugs to work out, but imagine the importance of this system once the vast amounts of information is imputed. Imagine the ability to scroll over a building in NYC and having the ability to click on it, which would bring up the building's history in regards to pictures, dates, building permits, and other historic information. Imagine having the ability to run a program that shows westward expansion to your children visually. Imagine having the ability to see the geographic changes of the world combined with personal stories and information of its residents. While the technology is a long way off as of now, but it is never too early to start. When you see something from a singular point of view, its like looking at life in 2D. However, looking at something from multiple points of view utilizing the full extent of knowledge, suddenly you are looking at life in 3D. Geographers have the ability to do there job without historians, however, I don't feel that historians can do there job without geographers. The importance of the vast data is too relevant to be ignored.
-Jason Barrett
Wednesday, September 4, 2013
Utilizing Technology in Research
As another week of reading has passed, the focus of the readings have shifted from questioning the validity of the digital revolution amongst historians to embracing it... kind of. The advantages of digital equipment when it comes to cataloging and copying is undeniable. Instead of spending hours copying information on a copier (and wasting a tremendous amount of trees), we can now take a picture of the page with its bibliography information and have it stored digitally. The benefits are simple, no more crowded offices with stacks of paper lying everywhere and an easy to recall system utilizing key words. To a younger crowd, this is studying 101, but to an older generation this may be a wake up call that today things can be a bit easier when one is willing to embrace the technology. Some people tend to dig there heels into the dirt when it comes to this, they genuinely find comfort in the stacks of books and the piles of paper littered with Post-It notes and page markers. While I myself have to admit that it is easier for me to flip through the papers and lay them out in an order on a table, I also must admit that spending the day at the copy machine isn't all that fun. There will be give and take, and there will definitely be an adjustment phase to utilizing the technology properly. Bear with me, it may not be as bad as you think.
In an online article ( http://www.historians.org/perspectives/issues/2005/0501/0501arc1.cfm ) authors Kirklin Bateman, Sheila Brennan, Douglas Mudd, and Paula Petrik take a 101 type approach to utilizing technology to your benefit when researching. This is a great article, what I find to be astonishing is that even though it was written in 2005, it seems to already be somewhat outdated. In the article it presses the reader to make an investment in a few items that will make their lives easier when it comes to using technology to their benefit while doing research. These items include a digital camera with several presets, a tripod, and a flat bed scanner that connects to ones laptop. These all make great sense and can be very helpful, however, a bit cumbersome. Why would I need all this equipment when there are cell phones such as the Nokia Lumia 1020 that houses a 41 megapixel camera? The technology of cell phones today, coupled with the use of apps such as Evernote, make researching and saving information so easy, that the idea of lugging around a camera, tripod, and/or scanner... much less hundreds of pieces of paper seems ridiculous. One big point that I found the article hits on is the importance of tagging information when you save it. One a normal computer when you save images you need to come up with a system that will allow you to easily recall the image. Such as putting the author of a book and the subject (i.e. Johnson_AndrewJackson), one possible downfall to this is simply forgetting your tag system that you have set up. Again, this problem can be fixed with Evernote, simply tag something "History" and Evernote will file it under history, it also reads authors names and digitally scans the words from the picture to make them searchable as well. So simply searching "Andrew Jackson Battle of New Orleans" under your "History" tags will bring up all saved pictures pertaining to Andrew Jackson and the Battle of New Orleans. Technology is not as hard and demanding as some make it out to be, this is the whole reason for apps, to make technology user friendly.
The question is, how do we catalog all of this data and use it to its full potential? The answer is simple, sharing. The problem is creating a system that this information can be shared on. While forums on the internet are an easy way for certified members to share information, the difficulty lies in the exhaustive searches that accompany these forums. Hours can be spent searching for a particular answer, so limitations must be set. High definition pictures can be a life saver when it comes to researching a particular piece of art or historic document when one can not readily fly across the country and make physical contact with the item. Although it is easy to research a document, wouldn't it be nice to have a website that one could go to and search for a high def version of a document? There are multiple sights that specialize in certain events or certain people, the Youtube effect can be very rewarding when it comes to research. The Youtube effect is simple, you research a topic and similar topics are brought up in a suggestion bar. These may be subjects that are associated with the searched topic, that an individual hadn't even thought to research. The amount of information on the internet is vast, and to organize this would be exhaustive, but I don't believe that the creators of Youtube ever realized that one day over 100 hours of video are uploaded every minute. Its possible that the history community is focused enough that a similar website could be created, if we could quit arguing long enough to get it going.
I'm beginning to really focus on my research project, and I must admit, it is a bit exhausting. I have a multitude of great ideas, the problem is figuring out how to effectively project them digitally without piggy-backing off of an existing idea. As with anything, it would be great to be the first at something. I'm strongly considering doing something that will tie into my internship at Hunt Cabin in the South Carolina Botanical Gardens. I've already done a documentary, but I think it would be awesome to go a bit deeper into what it takes to maintain and restore a historic building. I will update my progress on this as the days go by.
-Jason Barrett
In an online article ( http://www.historians.org/perspectives/issues/2005/0501/0501arc1.cfm ) authors Kirklin Bateman, Sheila Brennan, Douglas Mudd, and Paula Petrik take a 101 type approach to utilizing technology to your benefit when researching. This is a great article, what I find to be astonishing is that even though it was written in 2005, it seems to already be somewhat outdated. In the article it presses the reader to make an investment in a few items that will make their lives easier when it comes to using technology to their benefit while doing research. These items include a digital camera with several presets, a tripod, and a flat bed scanner that connects to ones laptop. These all make great sense and can be very helpful, however, a bit cumbersome. Why would I need all this equipment when there are cell phones such as the Nokia Lumia 1020 that houses a 41 megapixel camera? The technology of cell phones today, coupled with the use of apps such as Evernote, make researching and saving information so easy, that the idea of lugging around a camera, tripod, and/or scanner... much less hundreds of pieces of paper seems ridiculous. One big point that I found the article hits on is the importance of tagging information when you save it. One a normal computer when you save images you need to come up with a system that will allow you to easily recall the image. Such as putting the author of a book and the subject (i.e. Johnson_AndrewJackson), one possible downfall to this is simply forgetting your tag system that you have set up. Again, this problem can be fixed with Evernote, simply tag something "History" and Evernote will file it under history, it also reads authors names and digitally scans the words from the picture to make them searchable as well. So simply searching "Andrew Jackson Battle of New Orleans" under your "History" tags will bring up all saved pictures pertaining to Andrew Jackson and the Battle of New Orleans. Technology is not as hard and demanding as some make it out to be, this is the whole reason for apps, to make technology user friendly.
The question is, how do we catalog all of this data and use it to its full potential? The answer is simple, sharing. The problem is creating a system that this information can be shared on. While forums on the internet are an easy way for certified members to share information, the difficulty lies in the exhaustive searches that accompany these forums. Hours can be spent searching for a particular answer, so limitations must be set. High definition pictures can be a life saver when it comes to researching a particular piece of art or historic document when one can not readily fly across the country and make physical contact with the item. Although it is easy to research a document, wouldn't it be nice to have a website that one could go to and search for a high def version of a document? There are multiple sights that specialize in certain events or certain people, the Youtube effect can be very rewarding when it comes to research. The Youtube effect is simple, you research a topic and similar topics are brought up in a suggestion bar. These may be subjects that are associated with the searched topic, that an individual hadn't even thought to research. The amount of information on the internet is vast, and to organize this would be exhaustive, but I don't believe that the creators of Youtube ever realized that one day over 100 hours of video are uploaded every minute. Its possible that the history community is focused enough that a similar website could be created, if we could quit arguing long enough to get it going.
I'm beginning to really focus on my research project, and I must admit, it is a bit exhausting. I have a multitude of great ideas, the problem is figuring out how to effectively project them digitally without piggy-backing off of an existing idea. As with anything, it would be great to be the first at something. I'm strongly considering doing something that will tie into my internship at Hunt Cabin in the South Carolina Botanical Gardens. I've already done a documentary, but I think it would be awesome to go a bit deeper into what it takes to maintain and restore a historic building. I will update my progress on this as the days go by.
-Jason Barrett
Subscribe to:
Posts (Atom)