Wednesday 23 February 2011

Stock Exchange Stream Processing Simulator...

In this post i will demostrate how to write the world simplest tick data generator:

This command will write data to a file asx.txt:
-----------------------------------------------
$ while [ 1 ]; do echo $RANDOM >> asx.txt; done;

This command will read asx.txt in almost real-time:
---------------------------------------------------
$tail -f asx.txt -s 0.00001

you can write your strategies on this real-time data. This will just be like data coming from any data providing service/company.

Enjoy :-)

Monday 21 February 2011

Order by + Union All in oracle

In oracle you can not use order by with union all. They dont work together. In this post i will explain you a work around. We will cheat Oracle :-)

--------------------------
Create Table Foo(
Foo_Id Number,
Foo_Name Varchar2(20)
);

Create Table Bar(
Bar_Id Number,
bar_Name Varchar2(20)
);

Insert Into Foo Values(1, 'Fawad');
Insert Into Foo Values(2, 'Nazir');

Insert Into Bar Values(1, 'Mark');
Insert Into bar Values(2, 'Donald');

--Order by and Union All is not allowed together
Select * From Foo Order By Foo_Id desc
Union All
Select * From Bar Order By Bar_Name


--Work around (Cheating SQL :) )
select * from (Select * From Foo Order By Foo_Id desc) a
Union All
select * from (Select * From Bar Order By Bar_Name) b

Enjoy :-)

Round the number and always show decimal places...

The following is the way to round the number and always show two decimal places:

SELECT to_char(round(678,2), '9999999.99') FROM dual;

Friday 18 February 2011

DDL Rollback in Oracle...

Oracle does not support DDL Rollback, Try the following example:

CREATE TABLE FOO(
FOO_ID NUMBER,
FOO_NAME VARCHAR2(30)
);
ALTER TABLE FOO ADD FOO_DESCRIPTION VARCHAR2(50);
INSERT INTO FOO VALUES (1,'FIRST','THE FIRST ROW');
INSERT INTO FOO VALUES (2,'SECOND','THE SECOND ROW');

SELECT * FROM FOO;
ROLLBACK;
SELECT * FROM FOO;

Does rollback statement drops the table created as well? or undo the alter statement?

It only undo the Insert which is a DML, it can not rollbacl DDL statements.

There are two work arounds of this problem:

1. Using Oracle Flachback utility.
Problem with this solution is it has a lot of management overhead and does not work on statement level. This tool can rollback the whole table but can not rollback a simgle alter statement.

2. Database Change Management tools like dbdeploy & mybatis.
This tool can do it all and with quite good flexibility. However, if you already have a database and big tables and we want to drop the tables. In this case we will have to right large insert statements to undo the change.

3. Using both.
Looking at the shortcomings of the above two approaches, i would recommend using this approach.

UNION and UNION ALL in oracle

Difference between union and union all?

1. UNION only selects distinct values and UNION all selects all values.
2. UNION filters duplicates values and UNION ALL does not.

Simple example to test it:

create table foo(
foo_id number,
foo_name varchar2(30)
);

insert into foo values(1,'FAWAD');
insert into foo values(1,'NAZIR');

select * from foo
union
select * from foo;

select * from foo
union all
select * from foo;

Tuesday 15 February 2011

Java Regular Expressions

Ref: http://www.wellho.net/regex/javare.html

Java Regular Expressions - detailed manual


Summary of regular-expression constructs

Construct
Matches Characters
x
The character x
\\
The backslash character
\0n
The character with octal value 0n (0 <= n <= 7)
\0nn
The character with octal value 0nn (0 <= n <= 7)
\0mnn
The character with octal value 0mnn (0 <= m <= 3, 0 <= n <= 7)
\xhh
The character with hexadecimal value 0xhh
\uhhhh
The character with hexadecimal value 0xhhhh
\t
The tab character ('\u0009')
\n
The newline (line feed) character ('\u000A')
\r
The carriage-return character ('\u000D')
\f
The form-feed character ('\u000C')
\a
The alert (bell) character ('\u0007')
\e
The escape character ('\u001B')
\cx
The control character corresponding to x

Character classes
[abc]
a, b, or c (simple class)
[^abc]
Any character except a, b, or c (negation)
[a-zA-Z]
a through z or A through Z, inclusive (range)
[a-d[m-p]]
a through d, or m through p: [a-dm-p] (union)
[a-z&&[def]]
d, e, or f (intersection)
[a-z&&[^bc]]
a through z, except for b and c: [ad-z] (subtraction)
[a-z&&[^m-p]]
a through z, and not m through p: [a-lq-z](subtraction)

Predefined character classes
.
Any character (may or may not match line terminators)
\d
A digit: [0-9]
\D
A non-digit: [^0-9]
\s
A whitespace character: [ \t\n\x0B\f\r]
\S
A non-whitespace character: [^\s]
\w
A word character: [a-zA-Z_0-9]
\W
A non-word character: [^\w]

POSIX character classes (US-ASCII only)
\p{Lower}
A lower-case alphabetic character: [a-z]
\p{Upper}
An upper-case alphabetic character:[A-Z]
\p{ASCII}
All ASCII:[\x00-\x7F]
\p{Alpha}
An alphabetic character:[\p{Lower}\p{Upper}]
\p{Digit}
A decimal digit: [0-9]
\p{Alnum}
An alphanumeric character:[\p{Alpha}\p{Digit}]
\p{Punct}
Punctuation: One of !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~
\p{Graph}
A visible character: [\p{Alnum}\p{Punct}]
\p{Print}
A printable character: [\p{Graph}] or space
\p{Blank}
A space or a tab: [ \t]
\p{Cntrl}
A control character: [\x00-\x1F\x7F]
\p{XDigit}
A hexadecimal digit: [0-9a-fA-F]
\p{Space}
A whitespace character: [ \t\n\x0B\f\r]

Classes for Unicode blocks and categories
\p{InGreek}
A character in the Greek block (simple block)
\p{Lu}
An uppercase letter (simple category)
\p{Sc}
A currency symbol
\P{InGreek}
Any character except one in the Greek block (negation)
[\p{L}&&[^\p{Lu}]]
Any letter except an uppercase letter (subtraction)

Boundary matchers
^
The beginning of a line
$
The end of a line
\b
A word boundary
\B
A non-word boundary
\A
The beginning of the input
\G
The end of the previous match
\Z
The end of the input but for the final terminator, if any
\z
The end of the input

Greedy quantifiers
X?
X, once or not at all
X*
X, zero or more times
X+
X, one or more times
X{n}
X, exactly n times
X{n,}
X, at least n times
X{n,m}
X, at least n but not more than m times

Reluctant quantifiers
X??
X, once or not at all
X*?
X, zero or more times
X+?
X, one or more times
X{n}?
X, exactly n times
X{n,}?
X, at least n times
X{n,m}?
X, at least n but not more than m times

Possessive quantifiers
X?+
X, once or not at all
X*+
X, zero or more times
X++
X, one or more times
X{n}+
X, exactly n times
X{n,}+
X, at least n times
X{n,m}+
X, at least n but not more than m times

Logical operators
XY
X followed by Y
X|Y
Either X or Y
(X)
X, as a capturing group

Back references
\n
Whatever the nth capturing group matched

Quotation
\
Nothing, but quotes the following character
\Q
Nothing, but quotes all characters until \E
\E
Nothing, but ends quoting started by \Q

Special constructs (non-capturing)
(?:X)
X, as a non-capturing group
(?idmsux-idmsux)
Nothing, but turns match flags on - off
(?idmsux-idmsux:X)
X, as a non-capturing group with the given flags on - off
(?=X)
X, via zero-width positive lookahead
(?!X)
X, via zero-width negative lookahead
(?<=X)
X, via zero-width positive lookbehind
(? X, via zero-width negative lookbehind
(?>X)
X, as an independent, non-capturing group



Backslashes, escapes, and quoting

The backslash character ('\') serves to introduce escaped constructs, as
defined in the table above, as well as to quote characters that otherwise
would be interpreted as unescaped constructs. Thus the expression \\ matches a
single backslash and \{ matches a left brace.

It is an error to use a backslash prior to any alphabetic character that
does not denote an escaped construct; these are reserved for future extensions
to the regular-expression language. A backslash may be used prior to a
non-alphabetic character regardless of whether that character is part of
an unescaped construct.

Backslashes within string literals in Java source code are interpreted
as required by the Java Language Specification as either Unicode escapes
or other character escapes. It is therefore necessary to double backslashes in string
literals that represent regular expressions to protect them from interpretation
by the Java bytecode compiler. The string literal "\b", for example, matches
a single backspace character when interpreted as a regular expression, while
"\\b" matches a word boundary. The string literal "\(hello\)" is illegal
and leads to a compile-time error; in order to match the string (hello)
the string literal "\\(hello\\)" must be used.

Character Classes

Character classes may appear within other character classes, and may be
composed by the union operator (implicit) and the intersection operator (&&).
The union operator denotes a class that contains every character that is in at
least one of its operand classes. The intersection operator denotes a
class that contains every character that is in both of its operand classes.

The precedence of character-class operators is as follows, from highest to lowest:

1
Literal escape
\x
2
Grouping
[...]
3
Range
a-z
4
Union
[a-e][i-u]
5
Intersection
[a-z&&[aeiou]]


Note that a different set of metacharacters are in effect inside a character
class than outside a character class. For instance, the regular expression .
loses its special meaning inside a character class, while the expression -
becomes a range forming metacharacter.

Line terminators

A line terminator is a one- or two-character sequence that marks the end
of a line of the input character sequence. The following are recognized
as line terminators:

A newline (line feed) character ('\n'),
A carriage-return character followed immediately by a newline character ("\r\n"),
A standalone carriage-return character ('\r'),
A next-line character ('\u0085'),
A line-separator character ('\u2028'), or
A paragraph-separator character ('\u2029).

If UNIX_LINES mode is activated, then the only line terminators recognized
are newline characters.

The regular expression . matches any character except a line terminator
unless the DOTALL flag is specified.

By default, the regular expressions ^ and $ ignore line terminators and
only match at the beginning and the end, respectively, of the entire input
sequence. If MULTILINE mode is activated then these expressions match just after or
just before, respectively, a line terminator or the end of the input sequence
with the exception that the expression ^ never matches at the end of input,
even if the last character is a newline.

Groups and capturing

Capturing groups are numbered by counting their opening parentheses from
left to right. In the expression ((A)(B(C))), for example, there are four such groups:

1
((A)(B(C)))
2
(A)
3
(B(C))
4
(C)


Group zero always stands for the entire expression.

Capturing groups are so named because, during a match, each subsequence
of the input sequence that matches such a group is saved. The captured
subsequence may be used later in the expression, via a back reference, and may
also be retrieved from the matcher once the match operation is complete.

The captured input associated with a group is always the subsequence that
the group most recently matched. If a group is evaluated a second time
because of quantification then its previously-captured value, if any,
will be retained if the second evaluation fails. Matching the string "aba"
against the expression (a(b)?)+, for example, leaves group two set to "b".
All captured input is discarded at the beginning of each match.
Groups beginning with (? are pure, non-capturing groups that do not capture
text and do not count towards the group total.

Unicode support

This class follows Unicode Technical Report #18: Unicode Regular Expression
Guidelines, implementing its second level of support though with a slightly
different concrete syntax.

Unicode escape sequences such as \u2014 in Java source code are processed
as described in ?3.3 of the Java Language Specification. Such escape sequences
are also implemented directly by the regular-expression parser so that
Unicode escapes can be used in expressions that are read from files or from
the keyboard. Thus the strings "\u2014" and "\\u2014", while not equal,
compile into the same pattern, which matches the character with hexadecimal
value 0x2014.

Unicode blocks and categories are written with the \p and \P constructs
as in Perl. \p{prop} matches if the input has the property prop, while
\P{prop} does not match if the input has that property. Blocks are specified with the
prefix In, as in InMongolian. Categories may be specified with the optional
prefix Is: Both \p{L} and \p{IsL} denote the category of Unicode letters.
Blocks and categories can be used both inside and outside of a character
class.

The supported blocks and categories are those of The Unicode Standard,
Version 3.0. The block names are those defined in Chapter 14 and in the
file Blocks-3.txt of the Unicode Character Database except that the spaces are
removed; "Basic Latin", for example, becomes "BasicLatin". The category
names are those defined in table 4-5 of the Standard (p. 88), both normative
and informative.

Comparison to Perl 5

Perl constructs not supported by this class:

The conditional constructs (?{X}) and (?(condition)X|Y),

The embedded code constructs (?{code}) and (??{code}),

The embedded comment syntax (?#comment), and

The preprocessing operations \l \u, \L, and \U.

Constructs supported by this class but not by Perl:

Possessive quantifiers, which greedily match as much as they can
and do not back off, even when doing so would allow the overall
match to succeed.

Character-class union and intersection as described above.

Notable differences from Perl:

In Perl, \1 through \9 are always interpreted as back references; a
backslash-escaped number greater than 9 is treated as a back reference
if at least that many subexpressions exist, otherwise it is interpreted,
if possible, as an octal escape. In this class octal escapes must
always begin with a zero. In this class, \1 through \9 are always
interpreted as back references, and a larger number is accepted as
a back reference if at least that many subexpressions exist at that
point in the regular expression, otherwise the parser will drop digits
until the number is smaller or equal to the existing number of groups
or it is one digit.

Perl uses the g flag to request a match that resumes where the last
match left off. This functionality is provided implicitly by the
Matcher class: Repeated invocations of the find method will resume
where the last match left off, unless the matcher is reset.

In Perl, embedded flags at the top level of an expression affect
the whole expression. In this class, embedded flags always take
effect at the point at which they appear, whether they are at the
top level or within a group; in the latter case, flags are restored
at the end of the group just as in Perl.

Perl is forgiving about malformed matching constructs, as in the
expression *a, as well as dangling brackets, as in the expression
abc], and treats them as literals. This class also accepts dangling
brackets but is strict about dangling metacharacters like +, ? and *,
and will throw a PatternSyntaxException if it encounters them.

Monday 14 February 2011

Finding and Removing Oracle Jobs...

Find Oracle Jobs:

1. Find running jobs:

SELECT r.sid, r.job, r.this_date, r.this_sec, SUBSTR(what,1,40) what FROM dba_jobs_running r,dba_jobs j WHERE r.job = j.job;

2. Find jobs:
col job format 99999;
SELECT job, next_date, next_sec, failures, broken, SUBSTR(what,1,40) DESCRIPTION FROM all_jobs;

you can use user_jobs, dba_jobs, all_jobs as table option in 2, as required.

Removing a job:

exec sys.dbms_ijob.remove(_job_id_);

Recursively find all files and convert to unix format (excluding those in svn dirs)

Ref : http://snippets.dzone.com/posts/show/108

find . -type f \! -path "*svn*" -exec dos2unix {} \;

After svn checkout you should run this command in your local directory.

Thursday 10 February 2011

isNumber or isNumeric method for Oracle

Oracle does not have these functions, however you can use oracle regular expressions to achive this:

1. Example 1:
SELECT
*
FROM
_TABLE_NAME_
WHERE
REGEXP_LIKE(_COLUMN_NAME_,'^-?[[:digit:],.]*$')

This will return the _COLUMN_NAME_ values that are numeric.

2. Example 2:
select _COLUMN_NAME_,
case when regexp_like(_COLUMN_NAME_, '^-?[[:digit:],.]*$')
then
'Numeric'
else 'Non-Numeric' end as type
from _TABLE_NAME_

3. You can also use it as:
SELECT
*
FROM
_TABLE_NAME_
WHERE
NOT REGEXP_LIKE(_COLUMN_NAME_,'^-?[[:digit:],.]*$')
This will return the _COLUMN_NAME_ values that are non-numeric.

Saturday 5 February 2011

Command line for and while loop.. Shell Linux

Command line while loop:
$while [ 1 ]; do traceroute www.slac.stanford.edu; done;

Command line for loop:
$for x in 1 2 3; do ls -l; done

Enjoy :)

GUI based WinSCP & FTP for MAC.

Try:

Cyberduck
http://cyberduck.ch/

end-of-line character conversion: unix2mac dos2unix mac2unix

All these scripts are one liners:


"mac2unix"
cat $1 | tr '\r' '\n'

"unix2mac"
cat $1 | tr '\n' '\r'

"dos2unix"
cat $1 | tr -d '\r'

NOTE: "tr" (translate character) utility.

Friday 4 February 2011

Tuesday 1 February 2011

Problem running shell script in Cygwin

If you are getting these kind of error in cygwin:

# ./compare.sh
./compare.sh: line 2: $'\r': command not found
./compare.sh: line 5: $'\r': command not found
./compare.sh: line 7: $'\r': command not found
./compare.sh: line 9: $'\r': command not found
' for reading (No such file or directory) `./conf/compare.conf
' for reading (No such file or directory) `./conf/compare.conf
' for reading (No such file or directory) `./conf/compare.conf
' for reading (No such file or directory) `./conf/compare.conf
./compare.sh: line 14: $'\r': command not found

Its because you have to convert this file into a unix format. So use the following command:

# dos2unix compare.sh
compare.sh: done.

This will solve this problem

Azure OpenAI Architecture Patterns & Deployment Patterns

Sharing some useful links that will help customers architect Azure OpenAI solution using the best practices: (1) Azure OpenAI Landing Zone r...