Okay okay...
so a lot of you may remember not that long back asking for some help with my final year project, but at that stage I was just making enquiries so I could begin reading around the topic
However, today I had a meeting with my project tutor... this is the guy who has presented the problem for me to solve using some sort of program/application/web interface, so I was looking to him to give me some general pointers and shove me in the right direction
I spoke to him, and his face was blank as I explained the possibility of using activeX and ASP integrated into a webpage to create a nice interface, he had NO IDEA what I was talking about, and when I asked him what he would do he replied "I dont know the technicalities of it, I just want a solution to this problem"
anyway I was hoping you guys would point me in the right direction, as I dont have the amount of time to research into each and every possible method
here is the brief he posted to me
Quotehave no idea how many of my colleagues take the time to have a look at my web-pages. It would be nice if every now and then I would receive an overview, in adjustable detail, of all the users that have hit my page. From which external page did they come to mine? Which pages on my site where they actually looking at, and did they download any paper? how often have they visited my pages, recently? who are they, what do their home pages tell me?
I am quite stumped as to how to
gather the information/data as people are visiting the webpage, presenting it seems to be a simple case of creating an interface in ASP (or am i wrong???)
any help is massively appreciated
Quote from: JaitsuI am quite stumped as to how to gather the information/data as people are visiting the webpage, presenting it seems to be a simple case of creating an interface in ASP (or am i wrong???)
any help is massively appreciated
There are 2 ways to do this, the first way is to use what your web server logs. With some configuation apache or IIS can log the following:
* IP address of requestor
* Page they requested
* Referrer address (where they came from)
* The response code they got back (200, 404, 301 etc...)
* Amount of data transferred
* User agent (what browser the used)
Advantage of this method is that it will work with any site without any modifications.
However you might want more information - you might want to track an individual use via cookie for instance. In which case you will need some code on every page that does this and logs the details when they make a request.
This should be easy todo and you shouldnt spend much time on it - this is the easy part.
The hard part is turning these logs into something meaningful. Again there are a number of ways you could do this -
* Parse the log files and generate a bunch of static pages with the content in.
* Have something that reads and parses the logs on the fly (this will be slow)
* Have something that parses the log files and loads them into a database and have a front end which reads from that.
There are plenty of tools around that do the above already: awstats, analog, webalizer
fantastic, thanks for the reply
is it impossible to do with ASP for example?
this project is meant to be lasting my from now until may so I dont want to make it too simple, but obviously it needs to be realistic
would love to code the interface from scratch if feasible
Its perfectly possible to to it with ASP.
You could for instance have some ASP on every page that logged all the details you needed then write the interface in ASP.
Personally Id do it using Apache web server + php, but then thats just me :)
ahhh reet
well to be perfectly honest, i dont know ASP but i hear its relatively easy if you know VB, and i dont know PHP or Apache either
but obviously I would need an apache web server...
starting from scratch I just want something thats easy to understand and get a grasp of
Just use PHP as its easy to integrate with Apache and you could do this whole project in a day (if you knew what you were doing). That gives you an idea of how big (or not) this is really.
few hours tops...
Id not bother interfacing with IIS logs, Id collect the data myself on the page.
That said, you could sign up to google analytics, use the javascript and thatll do everything for you. There you go... youve found a solution to the problem.
Have to say that final year project appears to be more like a 1st year one. Also curious what computer science course fails to teach you about PHP, ASP, or what apache is? Thats fairly darn shocking.
This is not a few hour or even a day project. It is as long as you want to make it.
From my own experience I recommend adopting an "extreme programming" style where by you set out a very basic list of requirements, implement those then add more requirements.
Always be ready to refactor your code
Always make sure you have a version that works
Religiously make backups
Quote from: cornetThis is not a few hour or even a day project. It is as long as you want to make it.
From my own experience I recommend adopting an "extreme programming" style where by you set out a very basic list of requirements, implement those then add more requirements.
Always be ready to refactor your code
Always make sure you have a version that works
Religiously make backups
Not going to disagree that its as convoluted as you want to make it, but the question remains why would you want to. Part of your research for a project should be what current offerings are out there, as such you should have ruled out features and setups that are already available, or implemented in such a way to be commercially viable, and instead be looking to implement new features, or to perhaps increase the efficiency of some features, heck even to build upon a current system thats out there.
Sorry about this Jaitsu, Ive honestly no problem with you personally, but RTFM does come into it.
5 minutes googling would give you ideas as to how to go about harvesting information off of users.
From operating systems, to time spent on pages, to what people access, to what people did, where they came from, using javascript you may even be able to browse a users cookies to determine what their surfing habits are (dont quote me on that, I just dont get javascript :) can understand the code, but never needed to code from scratch with it)
XP != lifecycle.
Id go for an evolutionary lifecycle, which is what Cornet states :) Its associated with XP, and agile methods but you should be able to find out more information on it with a search for Evolutionary Software Lifecycle, also found within erm... think P. Somervilles Software Engineering, I think I may have version 4 or something here, but you should find it in your university library its a good book if you need details on software engineering practices.
Oh and aye make backups... oh christ make backups. I worked on a local file, which every 60 seconds would be uploaded to my uni ftp server, (both my windows and my linux accounts), and my own FTP server, as well as a copy to a USB disk after I completed any work on it, with a copy to CD-RW (handy stuff CD-RWs :)) every week or so.
^^jeeez thats a lot of backups
i understand the evolutionary lifecycle for software development, ive developed three systems in my time at university
two were pretty small scale databases, bit of advanced visual basic... however advanced it really can get (not very)
i did a project last year which was more advanced than this, involving developing a java interface from scratch to lay over an Oracle database (our accounts on the university servers prevented us from using the form wizards) which took an aaage to code, i think it was about 400 pages long in total
i think i may email the head of the module, because I dont feel that this project is all that big, from what you have all said and from what I have read
Quote from: Jaitsu^^jeeez thats a lot of backups
i understand the evolutionary lifecycle for software development, ive developed three systems in my time at university
two were pretty small scale databases, bit of advanced visual basic... however advanced it really can get (not very)
i did a project last year which was more advanced than this, involving developing a java interface from scratch to lay over an Oracle database (our accounts on the university servers prevented us from using the form wizards) which took an aaage to code, i think it was about 400 pages long in total
i think i may email the head of the module, because I dont feel that this project is all that big, from what you have all said and from what I have read
Well from what youve said youre course seems more focused on traditional software development rather than web development, fair cop. Explains the lack of teaching about php and asp :)
Well the thing is the project hes given you is a bit... well yeah its crap at least I think so, but thats because its scope can either be piss... or can be as grandiose as you want. Guess thats why hes given it to you its up to you to make it as complicated as you want, but youre going to have to do a lot of research to justify your contributions, at least thats my take on it. I designed my own project proposal, but I still based it on two pre-existing systems, I took the features of both, the disadvantages, and worked to bring the two closer together. Perhaps that is what he is looking for? In which case, youve got a shedload of reading ahead of you, as web statistics is a huge subject if you start thinking about it.
Youve got data collection, collation, and storage to figure out, and thats prior to even thinking about analysing the data youve harvested, and displaying it.
What is your project server running? thatll dictate your programming language perhaps. Or you could go the route of agents and masters, so processing scripts could be a socket or web service application itself. then you have agents coded in ASP, PHP, Ruby, Python, Perl, whatever... so it can work cross platform??
Just a few ideas for you.
You could make it good. but the premise as it stands is crap (and thats your project supervisors fault not yours).
Edit... if youve coded in JAVA using JDBC before, then perhaps webservices is your best bet, SOA is the big buzzword at present, and itd suit you for employment after too :), code in java, use tomcat and the associated tools to do the wsdl and shiznit for you, then you just have to code the interface to it :) easy... and by sheer nature of it, web services are language independent, so master can be java, agents can be whatever you want.
As for backups... hell yes.
Youve got to protect it, "sorry sir I lost my project" is no excuse, and you know even with those backups I had to rollback several times due to cock ups, including a caffine fueled all nighter resulting in me saving a blank file over my project file... thus deleting everything, thank god I had the 60 second FTP sync thing going on. (Used WSFTP for that I think).
Didnt this come up previously, and someone suggested that actually the interesting bit of this project wouldnt be the harvesting data part, but how you present the data.
Think someone suggested using the some of the Google Stats/Share price stuff as a guide for interesting things to do.
yes it has come up before
i posted again because I was expecting to receive SOME sort of directions from my supervisor
but he knows nothing about php, asp or apache
in fact, i asked him what he would do and he said he didnt know about the technical side of things
so god knows how he is meant to mark my development